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FOREWORD 



Many factors mediate the process of youth employabil ity development. Our 
prior National Institute of Education-sponsored research has examined the 
influence of significant others, part-time work experience, and vocational 
training. Conditioned by the knowledge gained from analyses of the influences 
of these factors, and, more important, conditioned by the wide knowledge base 
developed from thousands of person-hours of data collection from employers 
through direct observation and survey techniques, our research is culminated 
here appropriately with a focus on secondary school experiences. The central 
function of high schools is learning . Thus, how and under what conditions 
youth learn, or fail to learn, is the focus of this inquiry. More specifical- 
ly, the study addresses knowledge and skills resulting from the instructional, 
curricuiar, and contextual elements of high schools as they transfer to the 
workplace. 

This research program was directed by Dr. Kevin Hollenbeck. The primary 
author of this report was Dr. John Bishop, Associate Director of Research. Dr. 
Suk Kang contributed to the report. Project staff who capably performed the 
field data collection include Joseph Copeland, Margo Izzo, Dr. Floyd McKinney, 
Dr. Jennifer Humphreys-Cummings, Ruth Gordon, Angela Valentine, Tina Lankard, 
and Tom Tinkler. Professor Kathy Borman, of the University of Cincinnati, 
directed the data collection at one of the sites and provided valuable training 
assistance to the project staff. 

Numerous individuals at many schools provided helpful support. We would 
like to acknowledge in particular the assistance of Ray Bertelsen and Lee 
Whittaker, of the Westerville (Ohio) School System; William Dickinson of the 
Cincinnati School for Creative and Performing Arts (SCPA); Barbara Christen and 
Charles Gibson of Murry Bergtraum High School; and John Strebe of Mt. Hebron 
High School. Other individuals who graciously allowed us to observe their 
classrooms and/or provided interview time include Ms. Dell, Ms. Haas, Mr. 
Stull, and Ms. Rizzucidlo of SCPA; Mr. Spadero, Ms. Kern, and Mr. Zimmerman 
of Murry Bergtraum; Mr. Davis, Mr. Garrison, Mr. Doll, Dr. Odgers, Mr. Resch, 
Mrs. Bulthaup, and Mr. Debrose of Westerville; and Mr. Strebe, Mrs. Perle, 
Mr. McCrumb, and Ms. Gallager of Mt. Hebron. Finally, we would also like to 
thank the approximately 200 students who participated in our study by providing 
an interviews. 

We wish to express our gratitude to the National Institute of Education 
for funding this effort and to thank Dr. Ronald Bucknam, Project Officer. 

Reviewers of earlier drafts of this report— Professor Kathy Borman, Pro- 
fessor Thomas Long, Professor David Stevens, and Professor Robert Slavin—made 
many helpful suggestions. Student assistants who worked on the study were 
Jennifer Kling and Diane Stefan. The report was edited by Michele Naylor of 
the National Center's editorial staff; and it was typed by Debbie Fladen and 
Cathy Jones. 

Robert E. Taylor 
Executive Director 
The National Center for Research 
in Vocationl Education 



ERLC 



8 

V 



CHAPTER 1 



INCENTIVES, LEARNING, AND EMPLOYABILITY 
by John Bishop 

1.1 Introduction 

An individual's employability (perceived productivity in a job) is deter- 
mined by a constellation of skills, abilities, and traits both cognitive and 
noncognitive. Employability development is the process of developing or learn- 
ing these skills, abilities, and traits. Employability development is thus a 
particular type of learning. A theory of employability development is not, 
however, a special case of learning theory. 

Theories^ of learning tend to take what is learned as a given. What is at 
issue is how much of the material—whatever it might be--is learned. However, 
the list of things that might be learned is endless--breakdancing, reading, car 
repairing, and so forth--and, not all abilities contribute equally to employa- 
bility. Some abilities contribute to productivity in all jobs, others contri- 
bute to productivity in a much smaller set of jobs, and some make no contribu- 
tion to productivity. There is a limit on available learning time, so choices 
must be made. A theory of employability development must explain these 
choices. Why, for example, do some people choose to learn things that make 
them more employable and others elect to focus on abilities that have little 
reward in the labor market? 

A theory of employability development must also explain how much is 
learned. Why do some youngsters learn more mathematics than others? Why are 
some classroom management techniques mo^e successful than others? Why do peers 
support and encourage learning in some classrooms and not in others? One 
simple framework will be applied to all of these questions. 

Learning is a change that takes place in a person. It occurs when an in- 
dividual who is ready and able to learn, is offered an opportunity to learn and 
makes the effort to learn. All three elements are essential. 

Learning ability and readiness to learn depend on one's educational back- 
ground and intelligence. Background is important because many skills and abil- 
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ities are hierarchical (for example, B cannot be understood until A is under- 
stood so an individual's capability to learn B depends on whether they already 
know A). 

The opportunity to learn is in principle, open to every literate individ- 
ual. For most students, however, the costs are very high if teachers and text- 
books are not available. Thus, schools lower costs of learning and making 
school attendance compulsory lowers costs even further. When many individuals 
are taught the same thing at the same time, there are significant reductions 
in the per trainee costs of teachers. The learning opportunities, consequent- 
ly, come in packages called courses or modules. Sometimes they come in large 
packages with sequences of required courses called programs. Educators decide 
what courses to offer, what to include in a course, and how much material to 
cover. These decisions clearly have major effects on what students are exposed 
to and, through this mechanism, on what students learn. How large are these 
effects? What happens to learning when standards are increased? What influ- 
ences the effect of standards on learning? These are some of the questions to 
be addressed in this paper. 

The part of the learning equation that is controlled by the student is 
effort. Within the preset constraints of the opportunity, the student, along 
with his or her parents, chooses which school to -tend and which courses or 
programs to take. Students then further control their learning by regulating 
the effort they devote to each course. They decide whether to copy homework, 
whether to cut class, and whether to pay attention. 1 Studies in classrooms 
have found that the time engaged in learning is consideraoly less than the time 
available for learning (time not devoted to disciplinary or administrative 
matters). (Frederick 1977, 1979; Fischer et al, 1978; Klein et al 1979; 
Goodlad 1983; Halasz and Behm 1982). These studies have also found consider- 
able variation in the amount of available classroom time that students 
actually devote to learning. Time devoted to homework is also highly variable. 



*A physical or learning disability that prevents a student from paying atten- 
tion is considered to be affecting readiness and learning ability not effort. 
A key part of the distinction between effort and readiness is that effort is 
under the student's control whereas readiness is either exogenous or determined 
by previous decisions. 
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While 25 percent of sophomores report doing more than 5 hours of homework each 
week, another 20 percent report doing less than 1 hour per week. 

Even more important than the time engaged in learning is the intensity 
of the student's involvement in the process. After 2 years of studying Ameri- 
can high schools, Sizer (1984) concluded, M No more important finding has 
emerged from the inquiries of our stu<1y than that the American high school 
student, as student , is all too often docile, compliant, and without initia- 
tive 1 ' (p. 54) . Goodlad (1984) made a similar observation in noting, "The 
extraordinary degree of student passivity stands out." Student-teacher ratios 
of 18 to 1 imply that, in the aggregate, students spend nearly 18 times as 
many hours learning as teachers spend teaching. Student time is therefore a 
very important input into the educational process, and the intensity of use ot 
that time a very important determinant of learning. 2 

Readiness, opportunity and effort interact to produce learning. They do 
so in a very special way that is represented by the following equation: 

(1) Learningi = Li = f[Readinessi x (min(opportunityi , efforti)] 

where 1 i' indexes specific types of learning. 

In other words, learning depends on effort (E) or opportunity—whichever 
is the lesser. Improvements in opportunities to learn (for example, a new 
course or a change in curriculum) change learning only if effort is constrained 
by lack of opportunity or influenced by the cost (for example, tuition) of the 
opportunity. Changes in effort are, therefore, not an automatic response to 
changes in opportunity. The second key assumption is that the readiness or 
ability to learn varies across people and across competencies that might be 
learned. This means that it is efficient from both personal and societal 
perspectives for people to choose different mixes and levels of learning 
effort. 

The process of choosing learning effort . The stjdent is not just another 
input into an educational process that is controlled and directed by others. 
Rather, students are more appropriately viewed as entrepreneurs trying to grow 
and develop in an environment shaped by a great variety of forces: history, 

^Much of the impact of good teaching derives from its impact on the student's 
motivation to devote intense effort to learning. 

O 3 II 
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parents, teachers, peers, employers, gov-ernment, society, and one's own abili- 
ties. Students generally, and secondary school students especially, should be 
viewed as individuals freely choosing what to try to learn ana how hard to work 
at learning it. 

It is useful to model these choices as the outcome of a comparison of 
benefits and expected costs. The benefits are both inrrinsic--the joy of 
learning for its own sake— and extrinsic. The extrinsic rewards incluue the 
honor and respect that parents, peers, and tearhers give for achievement, the 
honor and respect given to those who receive the rewards that achievement 
brings (awards, higher pay, better jobs, influence, and fame), and the rewards 
themselves. The costs are time, psychic energy, money for tuition and books, 
loss of control over one's in-class time, and--frequent ly--the necessity of 
giving up old habits and ideas. 

The benefits ot learning are derived from five different sources. Total 
marginal benefits are, therefore, the sum of the following marginal benefits of 
learning:3 

(2) Joy of learning (BJ + 

♦ Value of parental recognition x increased recognition by parents (B p P L ) 

♦ Value of peer recognition x change in recognition by peers (B^) 

♦ Value of teacher recognition x change in recognition by teachers (BjT^) 

♦ Value of employer recognition x impact of learning on the pay and 

quality of one's job (YL> 

= Benefit of increased learning (dB*/dL) 

Recognition is everything that the individual or group does consciously 
or unconsciously to influence the student's choices. Such influence might take 
the form of explicit rewards or punishment, but is more likely to assume the 
form of subtle nonverbal cues and expectations that are seldom verbalized. 
Note that the impact of teacher recog-nition depends on both the marginal value 



3 The benefit of a specific level of skill can be represented mathematically 
by B* = B(L,P,A,T,Y). B(*) is the function that describes how utility depends 
on the amount learned (L), recognition from parents (P), peers (A), and 
teachers and the school (T); and labor market payoffs (Y). Differentiating B* 
with respect to L and choosing units for B* such that By = l, we have dBVdL = 
BL ♦ BpPL +BAAL ♦ BHL + YL* 
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that the student pUces on such recognition (By) and the degree to which 
teachers are aware of the learning that does occur and their propensity to 
reward improvements in learning (Tjj. The same holds for parental and peer 
recognition. 

The cost of learning (C) depends on two factors: the re'ationship between 

effort and learning (fE) and the cost of effort (CE)- Both of these factors 

are influenced by teachers, parents, and peers. Thus, the cost of an increment 

in learning is given by the equation: 

(3) dC _ r ,[: _ (^(opportunity , parents, teachers, peers) 

<lL 7^ fE(readiness, parents, teachers, peers) 

where Ce is the marginal cost of effort and f£ is the marginal impact of effort 
cn learning. 

In this framework, students with a knack for a particular subject learn 
more than others because their costs of learning are lower (that is, they can 
learn a given batch of material in less time) and because their achievements 
are more likely to be recognized and rewarded by others (an absolute 
achievement level is the basis of almost all rewards for achievement). 
Significant others—teachers, parents, peers, and employers--inf luence what 
students learn primarily through their control or influence over costs ar 1 
rewards. Students learn more from good teachers because. "costs". . .have 
fallen (that is, the material is better explained enabling the student to learn 
it faster) and ... "benefits". . .have risen (that is, the material seems 
intrinsically more interesting or the teacher gives praise and recognition to 
learning. ) 

Factors influencing the choice of 1 earning efforts . Schools and teachers 
influence employability development in a variety of w?ys, including the 
following: 

• They lower the costs of certain types of learning (for example, 
employment-related mathematics skills, typing, work habits) and 
raise the costs of learning subject matter or skills that are not 
a part of the curriculum. The schools control over the oppor- 
tunity set (curriculum) can have major effects on what is 
learned. 

• They directly influence the benefits of learning through the use 
of praise and feedback given in the classroom and the honor 
accor'd to particular subjects to study. 

• They define proxy measures of achievement (for example, tests, 
comments on themes, grades) and feed these back to the student 
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and parents. The high school transcript has a similar informa- 
tion transmission role for colleges and employers. Teachers' 
reactions to student contributions in class communicate similar 
information to peers. Such proxy measures can become extremely 
important motivators and have major impacts on student choices. 

• They influence the student's perception of the labor market and 
societal rewards for different kinds of learning. 

Peers can lower the costs of learning by studying together or teaching 
others and can increase the benefits of learning by recognizing and respecting 
others for their learning achievements. Employers influence the learning that 
goes on in schools by recognizing and rewarding it. 

Parents are generally thought to have the greatest influence over adoles- 
cents' decisions about schooling. Parent* help their children learn by lower- 
ing costs (for example, by serving as a tutor or forbidding alternative activi- 
ties until homework is done) or bv increasing the rewards (for example, by 
praising and honoring success at school). The attitudes of parents toward 
various forms of educational achievement are not written in stone, however. 
Parental attitudes respond to school policies and parential perceptions of the 
economic environment. Activities like interscholastic sports and sports 
banquets, in effect, honor the parents as well as the youth and thus strengthen 
parental incentives to promote athletic excellence. Plays, musical recitals, 
and evening assemblies recognizing academic achievement have similar effects on 
parental encouragement of afforts to develop these abilities. Thus, greater 
school recognition of learning can be expected to stimulate greater parental 
encouragement of learning. * 

If academic achievement was perceived to have no effect on labor market 
success, parents would probably be considerably less supportive of school 
achievement than they currently are. Doing well in high school has a much 
greater impact on labor market success in Japan than it does in the United 
states. That is why Japanese parents place such great stress on academic 
achievement and often pay for 10 to 15 hours a week of private tutoring. Thus, 
labor market rewards for academic achievement have pervasive effects on the 



4 In other words, ?i/ Tl > 0, which is the derivative of parental recognition 
with respect to schc and teacher recognition, is positive. 
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recognition that parents give to academic achievement. 5 Students do not even 
have to be aware of the laoor market to have their decisions influenced by it. 



There is considerable evidence that the economic rewards and prestige 
attached to various types of academic achievement influence student decisions 
about continuing in school and about selecting a field of study. College 
enrollment and student choice of field of study respond to economic payoffs, 
such as the income advantage and the perceived availability of jobs in a given 
fieid (Freeman 1971, 1976a, 1976b; Bishop 1977). Labor market conditions also 
affect a student's decision to drop out of high school (Bowen and F->r.egan 1969; 
Lerman 1972; Gustman and Steinmeier 1981). The minimum wage (Ehrenberg and 
Marcus 1982) and the quality of the schooling offered (Gustman and Pidot 1973) 
have been shown to affect drop out rates. The financial costs of school at- 
tendance have also been shown to have major effects. Numerous studies have 
shown that college attendance rates are increased when colleges are nearby, 
tuition levels are low, and generous financial aid is available (see Jackson 
and Weathersby 1975 for a review of the literature). 

The very simple framework outlined in the preceding section will be used 
to examine the following four important questions: 

• How does a youth's sense of self-efficacy influence his or her 
decisions about employability development? 

• What impacts do labor market signals have on a youth's decisions 
about employability development? Under what conditions are a 
youth's responses to these signals socially optimal? How does 
credentialing of educational achievement influence learning? 

• How do classroom systems of recognition for academic achievement 
effect learning? 

• How do peer group influences and systems of recognizing academic 
performance interact? When is it desirable to offer awards to 
the group rather than the individual? 

Each of these questions i: addressed in the following sections. 



5 In other words, P|_/ Y|_ > 0, which is the derivative of parental recognition 
with respect to the labor market rewards for learning, is positive. 
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1.2 Self-Efficacy 

The theory outlined above assumes that students choose their level of 
effort by calculating how much the extra effort will contribute to learning and 
how much specific learning will be rewarded. Since many of the rewards for 
learning will be realized in the future and learning is hard for the student to 
measure, decisions about effort must be based on anticipated rather than actual 
relationships. It is quite possible for beliefs to differ significantly from 
reality. The belief that one's effort, not fate or chance, determines one's 
success is called internal locus of control or self-efficacy. Numerous studies 
have found a positive correlation between feelings of self-efficacy and 
learning. Table 1 presents data from Coleman et al. (1966) concerning the 
verbal achievement of 9th grade students categorized by their reaction to the 
following statement: "Good luck is more important than hard work for 
success. " 

TABLE 1 

SCORES ON A VERBAL ACHIEVEMENT TEST 
BY 

THE STUDENT'S SELF-EFFICACY BELIEFS 





Good Luck 


Hard Work 


White-North 


45.4 


54.8 


Wh its-South 


42.9 


52.5 


Black-No'th 


40.0 


47.1 


Black-South 


36.6 


43.3 


Hispanic 


38.6 


46.2 



SOURCE: Date from Coleman et al. 1966, p. 323. 



The students wlto felt that hard work determined success had considerably great- 
er verbal achievement scores (54.8 rather than a 45.4 for whites from the 
north). This relationship remained strong when family background and self- 
esteem were entered into the model to compete with self-efficacy in explaining 
verbal achievement. 

Part of the reason for this relationship is that successful learning 
causes the student to feel more in control of his or her fate. Another reason 
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for the relationship is that a belief in the efficacy of effort results in 
greater effort and more learning. Hotchkiss's (1984) analysis of changes in 
test scores between sophmore and senior year found that one's sense of control 
over one's fate measured when the student was a sophmore had an important 
effect on learning. Hotchkiss also found that causation runs the other way as 
well. Changes in beliefs abou+ controlling one f s fate were positively affected 
oy verbal achievement measured when the student was a sophomore. 

The formal theory presented in chapter 2 predicts that when rewards are 
based on a pass-fail criterion, the student will give up trying to learn. 
Dropping out responses like these have been uncovered in laooratory research on 
both animals and people and are often called learned helplessness (Seligman 
1972, Miller and Seligman 1973, 1975, 1976). Tht decision to withdraw from 
competition can be quite rational when the cutoff score for passing is quite 
high or when scores which determine whether one passas have a large random 
element. 

1.3 Ubor Market Signals 

A critical aspect of employabil ity development is to build information 
about the performance demands of the labor market into the incentive structure 
of schools and other training institutions. To do this entails a two-direc- 
tional flow of information. Students must know what knowledge, skills, and 
dispositions are rewarded in the labor market; employers must be able to deter- 
mine which individuals exhibit the characteristics that make them productive 
workers in specific jobs. The nature of this information flow helps to deter- 
rrnna what is learned, how much is learned, and how well it is learned. 

A complete theory of information flow between students and the labor mar- 
ket must answer three questions: (1) What information should be communicated? 
(2) What incentive structure will best motivate students to act on information 
they receive? (3) What rate of information flow is optimal? 

The first question has to do with identifying characteristics that enhance 
empl oyability. It is information about employabil ity characteristics that 
should be communicated. The issue is as follows: Given the extreme variety of 
jobs and their corresponding tasks, what constellation of knowledge, skills, 
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and attitudes can best be learned at school, at other training institutions, 
and at work? The seco.,^ question assumes as given the content of what is 
learned. The issue here is how to get students to learn the necessary 
material. The final question raises the issue of relative costs and benefits 
of information. Communication is costly; therefore, in determining the rate of 
communication, it is important to identify an optimum that balances costs 
against benefit. 

The theoretical work presented in this report focuses on question 2. In 
order to show the relevance of this work to labor market signals, the theory is 
applied to current circumstances in the U.S. labor market. Current conditions 
are summarized first, then theory is used to assess those conditions. 

Bishop (1985) identifies the following four key features of the connec- 
tions between school performance and employment outcomes in the United 
States: 



t Employers obtain little or no information about job applicants 1 
grades, performance on standardized tests, deportment in high 
school, or participation in extracurricular activities. 

• School referrals are insignificant in locating jobs for job 
applicants. 

• High school grades and tests scores have little or no impact on 
the employment success of youth who have not attended college. 

• The effects of cognitive abilities learned in school on employ- 
ment success that do occur are not evident until an individual 
has accumulated several years of experience in the labor market. 

Typically, the only information requested from job applicants regarding their 
schooling is number of years completed, the highest degre, earned, and the area 
of specialization (if any). 

Several aspects of learning theory as presented if. this report suggest 
that these features of the information flow between the labor market and stu- 
dents do not generate strong incentives for learning. The most obvious diffi- 
culty is that there is no direct communication of what is learned in school 
to employers. As a result, there is little reward for learning in the (non- 
college graduate) labor market, and those rewards that dQ accrue do so over a 
lengthy time period. 
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Still, some trickle ot information does get passed on to employers through 
various formal and informal channels, and some modest reward fcr learning is 
observed in the labor market. But theory suggests that the current situation 
is highly ineffective. There are several reasons for this. First, there is a 
high degree of uncertainty associated with labor market incentives for learn- 
ing. Learning theory indicates that uncertainty discourages effort. Second, 
the importance of a diploma in certifying employability is analogous to a 
pass-fail testing system with a very low level of difficulty at the critical 
point of failure. Theory indicates tuat this incentive structure will give 
strong motivation only to those who are moderately below average in ability. 
High ability youth can pass (ea r n a diploma) with minimum effort, and very low 
ability youth are motivated to drop out because their chance of success is too 
small. Th'rd, labor market payoffs to learning accrue so slowly that individ- 
uals with strong preferences for income now rather than later See little reason 
to exert effort in learning. Since many high school students probably have 
relatively short time horizons, labor market payoffs provide poor motivation 
for them. Finally, the incentives in the labor market are analogous to school 
evaluations in which all students are judged against the same criteria. Thus, 
the advantages in motivating students that occur when each student is evaluated 
against his or her own baseline ability are clearly not present in the labor 
market. 6 In summary, the theory developed in this report suggests that the 
U.S. labor market provides a poor incentives for learning in our nation's 
schools. 

1.4 Student Reward Systems 

Motivating students to try harder is one of the primary objectives of 
classroom reward systems. Classroom rewards are sometimes graduated (for 
example, a numerical grade on a test) and sometimes discrete (a passing grade 
versus a failing one). Which of these forms of feedback and reward is more 
effective at stimulating effort? If a pass-fail system is used, where should 
the passing criterion be set? Which system is least likely to induce a student 
to drop out? 



^To a degree, the concept of matching individuals to jobs could be viewed as 
a handicapping system, but this is a rough analogy at best. 
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Recognition can be based on an absolute measure of performance, on pro- 
gress since a previous test, or on performance relative to an individualizec 
target (handicap). Which system stimulates effort the most? Which system 
minimizes the propensity of the least able students to give up on the task of 
learning (to dropout)? If handicaps are used, how should they be formulated? 

Rewarding progress or rewarding achievement . We begin by examining 
whether recognition should be given for an absolute level of achievement or for 
growth over an individualized baseline performance level. The primary purpose 
of recognizing and rewarding learning achievements is to induce students to 
achieve all that they are capable. Most of the recognition for academic 
achievement, however, goes to a very few students who are at the very top of 
some absolute scale of achievement. Although such rewards may challenge the 
most talented students to achieve, they are out of the reach of most students 
and hence do not influence their effort. In fact, in many schools, the 
majority of students see these prizes and the values they represent as threats 
to their own self-esteem. Their reaction is often to denigrate the reward and 
the learning it represents and to honor other forms of achievement— athletics, 
break dancing; or being "cool, 11 attractive, or popular—which offer tiiem better 
chances of success. In some cases, the denigration of learning takes the form 
of cheating or acting up in class and thus, may disrupt the learning of 
others. 

Teachers sometimes try to deal with this problem by praising students who 
try hard and by withholding praise from students who do not. There are limits 
to this approach, however, for teachers are not able to observe the true in- 
tensity of a student's effort and, as a result, may reward only the visible 
proxies of effort. In addition, peer group norms often stand in direct opposi- 
tion to the teacher's norms. Often, peer norms call for getting by with a 
minimum of effort; therefore, being singled out by a teacher as someone who 
tries hard may hurt one's reputation (Covington and Beery 1976). As Thomas 
(1980) has put it, "Low effort has great survival value for many students. It 
provides an excuse and a way to avoid ascriptions of low ability under failure 
conditions and can serve to aggrandize estimates of ability when success oc- 
curs" (p. 215). 
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There are really only two kinds of reward structures that avoid these 
problems. The first system organizes competition so that it occurs among stu- 
dents of approximately equal ability and makes awards to the winners in 
each league. 7 The second system recognizes and rewards the achievement of 
individual learning goals tailored to the student's ability and background. 
The individualized goal must be achievable, but not be so easy that the student 
does not have to make an effort to achieve it. 

In chapters 2 and 3 of this report we present a formal analysis of reward 
systems that recognize achievement in excess of an individualized learning 
goal. The main findings of the analysis are summarized and explained in this 
chapter without the use of mathematics. 

The assumptions that lie behind the analyses are the following: 

• Learning requires effort. 

• Effort is costly. 

• The more a student tries to learn in a given time period, the 
greater is the marginal cost of additional learning. This oc- 
curs because there are diminishing returns to increases in ef- 
fort and because the marginal costs of effort increase as effort 
increases. 

• The test that is the basis for distributing rewards should mea- 
sure the skill and knowledge that are the objectives of the 
course. The test need not De a perfect measure of achievement of 
learning objectives; however, it is important that errors in 
measurement be random and not too large. (If the test measures 
only some of the learning objectives and sttdents are aware of 
this, rewarding performance on the test will tend to shift stu- 
dent effort toward only those learning objectives that are in- 
cluded on the test. ) 

o Students have beliefs about the magnitude of the average re- 
lationship between learning, effort, and the cost of effort and 
about the variability of test scores given their effort. Test 
score outcomes have a random element partly because the tests are 
not perfectly reliable measures achievement of learning objec- 
tives and partly because students are not able to anticipate ex- 
actly hew much learning their effort will produce. 



'In team sports, this restructuring of competition is common. High schools 
of thp same size typically compete against each other in state tournaments and 
exttd mural softball and soccer leagues are commonly organized into divisions. 
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Not surprisingly, the model predict that an increase in the reward for 
learning will generate an increase 4 rort. The model also predicts that 
combinations of rewards for those who earn the most and penalties for those 
who do not learn also increase effort. Our focus is on the effects of other 
environmental factors such as test reliability, feelings of self-efficacy, and 
reward structure; therefore, the analyses is conducted under the assumption of 
a fixed budget for rewards and no penalties. A somewhat less obvious 
prediction is that the reliability of the tests used to measure learning (and 
student beliefs about the importance of chance in determining test outcomes) 
can have important effects on the effort a student puts into learning. If 
rewards are based on a pass-fail criterion that has been set at a point 
designed to elicit maximum effort from the student, less-reliable tests will 
reduce the degree to which success (passing) depends on effort and thus cause 
effort to be reduced. The same result can occur when the amount of reward is 
scaled to performance in a linear fashion (as with grades and numerical test 
scores), if students are risk averse, or if there are floors or ceilings on the 
linear reward system. This implies that teachers may increase the effort that 
students put into their studies by using the most reliable tests possible and 
by convincing students that effort, not chance, determines their test score. 

Another issue addressed in the analysis is the setting of standards. As- 
suming that the error in measuring performance is normally distributed and that 
the costs of an extra unit of learning are increasing at a constant elasticity, 
the greatest amount of effort is elicited when each student perceives about a 
60 to 90 percent chance nf passing provided he or she puts out the optimal 
level of effort. Up to a point, raising the criterion for a passing grade 
raises the effort that students put into learning. For the assumptions speci- 
fied in table 2 (see chapter 2) the increase in learning is equal to about two- 
thirds of the increase in the passing standard. This means that the failure 
rate rises as the passing standard increases, but not by nearly as much as one 
might anticipate if one assumed effort does not respond to the level of the 
passing standard. These results depend on how rapidly the marginal costs of 
learning rise and on the variance of the test score. 

One of the most important findings of the analysis was that pass-fail 
reward systems have very different effects on the amount ana 'istribution of 
learniny than do graduated reward systems. When rewards are a linear function 




of learning, the faster learners experience a higher payoff tc learning than 
slow learners and so they put out more effort than slow learners. While slow 
learn-ers put out less effort they always put some effort into their study. In 
a pass-fail reward system, a single pissing standard for everyone has the 
effect of inducing the moderately slow learners to work harder than the fast 
learners but it causes the very slowest learners to drop out all together. 

Establishing a pass-fail system that has different passing standards for 
students with different readiness and ability can forestall dropping out and 
can induce ev3ryone— slow and fast learners—to try harder. Graduated reward 
systems that reward performance above an individualized learning expectation 
are also more efficient than graduated reward systems that reward all learning 
above a threshold of zero. 

The empirical evidence . Four experiments have been conducted to evalu- 
ate the effect of classroom reward systems based on individualized learning 
expectations ( I LE ) . In three of the four studies, there were large positive 
effects on student learning. 

In the first experiment, Slavin (1978) used a 2 x 2 factorial design to 
study the effects of team versus group rewards for performance and the effects 
of comparison of students within ability-homogeneous groups versus comparison 
of students with the entire class. The theory outlined previously suggests 
that competition elicits the greatest effort when competitors are evenly 
matched (Lazear and Rosen 1981). Consequently, when students compete for 
individual rewards, it is important, that they be evenly matched. When teams 
compete for group rewards, it is important that the teams be evenly matched. 
When team scores are an average of individual scores, it should not matter 
whether scores are scaled by comparison with the entire class or scaled by 
comparison with other students of similar ability. If the competing teams ar^ 
evenly matched, winning depends on everyone's trying, regardless of how scores 
are scaled. Consequently, one would expect the nature of comparisons (whether 
it is with equals or with the entire class) to affect performance only when 
recognition is based on individual performance. 

A total of 205 seventh-grade students (all but 3 of whom were white) in 8 
English classes participated in the experiment. Four of these classes did not 
group students into teams, but instead rewarded students for their individual 



performance. Rewards went to the winners of a competition amorg students of 
relatively equal ability in two of these four classes. A weekly class news- 
letter recognized those who had done well in their divisional competition for 
high scores. Students received standard percentage scores on their quizzes in 
the other two classes. The experiment manipulated a supplementory reward 
system (recognition in a class newsletter); it did not change the formula for 
calculating report card grades. 

The divisional competition and the newsletter announcing the results 
raised time on task from 73 to 82 percent, a statistically significant in- 
crease. It did not, however, produce a statistically significant improvement 
in performance on either the standardized or curriculum-specific achievement 
measures. 

The secontf experiment (Slavin 1980) involved 10 teachers and 16 English 
classes containing 387 sixth, seventh, and eighth graders. Whenever possible, 
teachers taught both experimental and control classes, and random assignment 
was used to determine which classes were assigned to the ILE treatment. All 
students studied the same 9-week language mechanics unit and covered the mater- 
ial on the same schedule. The only difference between treatments was in the 
use made of the students 1 scores on biweekly quizzes. In the ILE treatment, a 
baseline score was established for each student based on a 50-item pretest. 
The base score was set 10 points below the students' expected summed scores on 
the two weekly quizzes, and bonus points were awarded for exceeding the 
base.8 

Achieving the base score or exceeding it by 1 or 2 points earned 1 bonus 
point. Exceeding it by 3 to 5 points earned 2 bonus points and so on. Achiev- 
ing a perfect paper or exceeding one's base by 18 points or more earned a 
student a maximum of 7 bonus points. New base scores were calculated each week 
to reflect more recent performance and to correct for possible unreliability of 
the pre-test. The new baseline score was a weighted average of the latest 
score (wt s 1) and the last week's baseline plus 10 (wt * 3) minus 10. Slavin 
described the experiment as follows: 

8 The problem of variability in test difficulty from week to week was handled 
b> adjusting all quiz scores to a standard class median of 40. If, for exam- 
ple, the true class median was 32, 8 points were added to each score. 
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Students were told that their task was to exceed their base scores 
by as much as possible. Each week, students were informed of their 
base scores, the degree to which they exceeded them, and their 
[plus] bonus points, A weekly class newsletter listed the students 
who earned 5, 6 or 7 plus points, . . 

In the control classes, students studied the same material on the 
same schedule but they simply received grades on their quiz scores. 
They did not receive base scores, plus points or newsletters • . • 
All students . . . received 6-week grades based on their actual 
quiz scores (not their plus points) and their other English-related 
activities, such as literature apd composition. Thus, the reward 
system used in the experimental group was a supplement to, not a 
replacement for, traditional grading (Slavin 1980). 

This rather mild intervention produced a statistically significant in- 
crease in student performance on the posttest that was equal to 42 percent of 
the pooled class-level standard deviation for the posttest. This experiment 
demonstrates that a remarkable gain in achievement can be obtained by simul- 
taneously changing the way students receive feedback on their performance 
(emphasizing plus points in quiz grading rather than raw scores) and giving 
greater recognition to successful learning (publishing a newsletter). 

Since it might have been the newsletter rather than the ILE grading that 
caused the improvements, more experiments were necessary. The third experiment 
(Beady, Slavin, and Fennessey 1981) involved an ILE system similar to the one 
described previously, except that (1) the baseline recalculation was somewhat 
more sensitive to recent performance, (2) there was no adjustment for week-to- 
week variations in the difficulty of the quiz, anu (3) the seven top performers 
were recognized by certificates rather than through a newsletter. Beady and 
his co-workers described the procedure as follows: 

The comparison group studied the same math curriculum using the 
same focused schedule of teaching, worksheets and quizzes . . . 
However, instead of receiving points in comparison to base scores, 
these students received individual percentage scores, and the seven 
highest scoring students received the same certific f .es given the 
ILE students (Ibid 1981 p. 16). 

This intervention produced a slightly smaller gain (about 30 % of a stan- 
dard deviation) on the posttest. Even though this gain was quite respectable, 
it did not represent a statistically significant difference from zero because 
the sample size (6 classes and 180 students) was considerably smaller than in 
the previoi;: experiment. 
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Beady, Slavin and Fennessey (1981) interpreted the lack of a significant 
effect as not supporting the findings of Slavin (1980) regarding the beneficial 
effects of basing rewards on improvement rather than on relative standing, and 
they spent a number of paragraphs speculating on why there was a difference 
between the two studies. In fact, however, the estimated effects of ILE in the 
two experiments are not significantly different. If the initial hypotheses had 
been that effects were equal to 42 percent of a standard deviation, it would 
not have been rejected. 

The subjects of the fourth experiment were 248 seventh- and eighth-grade 
students in 10 mathematics classes in a desegregated Baltimore middle school. 
The treatments were identical to those used in the 3rd experiment described 
previously. Both the ILE classes and the control classes used a focused 
schedule of instruction for 10 weeks. Both received certificates; however, in 
the ILE class, the certificates were based on improvement whereas, in the 
control group, they were based on relative standing in the classroom. The ILE 
treatment was found to have a particularly positive effect on blacks, and for 
this group, the treatment was statistically significant. The gain in achieve- 
ment attributable to ILE was about 24 percent of a standard deviation (on the 
posttest) for blacks and about 12 percent of a standard deviation for whites 
and blacks together. 

Slavin (1983) has concluded that "use of equal opportunity scoring may 
enhance the achievement effects of cooperative learning, but the evidence is 
not yet conclusive," (p. 72). We agree with this assessment when students are 
grouped in heterogeneous teams and are rewarded for group performance. When 
equally matched groups compete every team member's performance counts; there- 
fore, fnotivation is strong even when individualized learning expectations are 
not used. ILE appears to have important (20 percent of a standard deviation is 
important in our view) effects on learning when rewards go to individuals 
rather than teams. 

The ILE reward systems tested in these experiments, however, had updating 
rules which made the baseline very sensitive to one's performance in the last 
few weeks. In chapter 3, we show that pure ILE reward systems with an updated 
ILE invite students to lower their baselines by purposely doing poorly at first 
and then surprising the teacher with a great improvement. If all rewards are 
based on the ILE, the down-up strategy would be the way for most students to 
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maximize rewards over the course of the experiment,. Quiz scores were probably 
a component of the fins 1 grade in most of the ILE classes, so it is unlikely 
that students would do poorly on purpose. Nevert leless, the rapid updating of 
the ILE baselines does significantly reduce the incentive to do well on the 
quizzes in the early part of the marking period. 

The rationale for updating the ILE on the basis of later quiz scores is 
the need to improve the accuracy of the student's ILE. The test given at the 
beginning of the year that is used to construct the ILE is not a perfectly 
reliable measure of the student's ability, so the initial ILE is set too high 
for some students and too low for others. This initial classification error is 
eliminated over time by incorporating additional quiz scores into the ILE. The 
result is an ILE that is a more accurate reflection of the student's ability 
and an ILE will as a result be seen as fair. This is a legitimate argument for 
updating the ILE t but is is also an argument for greater care i n assigning the 
initial ILE. Initial ILE's should not be based on one pre-test. The teacher 
should obtain the student's scores or iptitude tests and standardized achieve- 
ment tests in the subject (or a related subject) being taught, and grades in 
the same subject in previous years. In addition, the first class period of the 
year should be devoted to a 50 mi mil. placement exam that covers both the 
material from previous years and the material scheduled to be covered in the 
class. The test scores and grades could then be combined into a single index 
by calculating Z scores and then constructing an average. 9 With more 
accurate initial ILE's, the updating of the ILE's on the basis of recent 
quizzes, could be postponed until the end of the first or second month and the 
sensitivity of the ILE to recent quiz scores reduced by reducing the weight 
assigned then i ■ the calculation of the ILE. 

The analysis predicts that an ILE system that uses a reliable ability test 
to establist the initial ILE for each student and then sticks with that base- 
line (or adjusts very slowly to more recent performance) will generate con- 
siderably larger increases in learning than an ILE that is rapidly updated. 
Experiments that contrast an updated ILE with a f^d ILE would be very help- 
ful. 



^Another approach to normalizing the scores so that they can be combined 
involves calculating the means of the top and bottom quartile, dividing the 
score or grade by the difference betwen the two Means, and then averaging them. 




1.5 Peer Influences 

A respected creative writing teacher in a highly-rated school district 
recently advised a 6th grader not to volunteer to answer questions when class- 
mates did not volunteer. The explanation he offered was that vo\. Peering too 
often would make the student unpopular. 

Why are the students who study hard and excel in class often rejected by 
their peers? One reason, already discussed, 1s that since more recognition is 
limited to a few, most students conclude they cannot be winners in the academic 
arena. Consequently, they set other goals for themselves and denigrate academ- 
ically successful students. 

A second reason is that most classrooms are competitive environments in 
which one student's success reduces the chances of others. Grades are based on 
a curve. Awards are given for rank in class rather than for achieving a fixed 
standard of excellence. If everyone in the class were to improve, the group as 
a whole would not benefit. This is because the total number of awards is 
fixed. 

Adolescents care a great deal about their reputation with peers. To them, 
the rewards teachers offer--grades, a smile, verbal praise, a project pinned 
upon a bulletin board--seem small in comparison with the rewards that their 
peer group is able to bestow—friendships, social acceptance, and popularity. 
Peer norms that reflect the interests of the group develop naturally. Actions 
that bring honor to the group as a whole, such as outstanding performance 1n an 
athletic competition, the orchestra, or a play, are respected and heavily 
rewarded by peers. 10 Efforts to win a zero-sum competition such as studying 
hard for a test or volunteering in class are actively discouraged by peers. 
Activities which do not affect one's immediate peers, such as studying for an 
SAT or a statewide Regents exam, reading War and Peace for the fun of it, or 
competing in a science fair receive better treatment from peers; however, peer 
pressure still works against sucn activities because they compete with peer 
group priorities regarding the use of time. 



1U In one school we visited, for instance, each member or the football team 
received a visit and a present of cookies or something similar from a member of 
the pep club the night before each game. 
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How can schools and teachers get peei s to value academic effort and learn- 
ing? One possible approach is to reward the group for the individual learning 
of its members. This is the approach taken in cooperative learning. Students 
are grouped into evenly matched teams of 4 or 5 members that are heterogeneous 
in ability. After the teacher presents new material, the team works together 
on worksheets to prepare each other for the periodic quizzes. The team's score 
is an average of the scores of team members, and high team scores are 
recognized in a class newsletter or through group certificates of 
achievement.il 

Slavin (in press) has recently reviewed 27 field experiments that compared 
cooperative learning strategies combining group study and group reward for 
individual learning with the standard individual-reward-for-individual-learning 
system (Slavin 1985). 12 In 24 of these studies, cooperative learning had a 
statistically significant positive effect on learning. Where effect sizes were 
available, they were approximately 30 percent of a standaro deviation on the 
posttest. 

A number of studies have been conducted in which the various components of 
the cooperative learning model described previously have been tested on their 
own or in 2 x 2 factorial experimental designs. The four studies that examined 
the effects of group study without group rewards for individual lear.ing found 
that sue! a strategy had no positive effects. Group study methods that offered 
group rewards based on the quality of a group product were also not found to 
increase learning. These results suggest that the two key ingredients for 
successful cooperative learning are as follows: 

• A cooperative incentive structure—awards based on group 
performar* e--seems to be essential for students working in groups 
to learn better. 

t A system of individual accountability in which everyone's maximum 
effort must be essential to the group's success and the effort 

n In many cooperative learning systems, the individual's contribution to the 
team score is a gain in score relative to an individualized learning 
expectation. 

12 The review was limited to studies in which treatments lasted at least 2 
weeks in a regular school setting. The experimental and control groups were 
exposed to the same curriculum, and students were not allowed to help each 
other on final tests. 
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and performance of each group member must be clearly visible and 
quantifiable to his or her group mates. 

These results provide important evidence of the importance of peer norms. 
What seems to happen in cooperative learning is that the team develops an iden- 
tity of its own, and group norms arise that are different from the norms that 
hold sway in the student's other classes. The grojp's identity arises from the 
extensive personal interaction among gi^up members in the context of working 
toward a shared goal.* Since the group is small and the interaction intense, 
the effort and success of each team member is known to his other teammates. 
Such knowledge allows the group to reward each team member for his or her 
contribution to the team goal, and this is what seems to happen. This is the 
prediction of the theoretical examination of cooperative organizations pre- 
sented in appendix C, which addresses the issue of how a cooperative group will 
distribute the rewards it receives to its members when the sum of the indiv- 
idual efforts of its members determines the magnitude of the reward received by 
the group. The conclusion of the analysis is that the sharing rule adopted by 
the cooperative organization will take into account the effort and contribution 
the member made to the group goal. When the output is a simple sum of individ- 
ual outputs (as in cooperative learning without task interdependence), there is 
a c -for-one passing through to group members of the reward contingencies 
faced by the group. 

Our interviews with students on cooperative learning teams suggest that 
this prediction of theory is correct. Team members do encourage each other to 
try harder. One 9th-grade boy who found math difficult said the following: 

Student 1 : He [the teacher] gives me all these problems and they 
[team members] say you better get them right and I can't. 

Interviewer : So what happens when you say that [I can't do the 
problems]? 

Student : They tell me not to give up. 

Interviewer : How does that make you feel? 

Student : Makes me want to put more effort into it. 

Another student was asked if he had had an opportunity to teach or help any- 
one in his group. He answered as follows: 
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Stu dent ; Yes, I've helped some people but some people start to 
give up and that doesn't help. 

Interviewer : What do you say to a team nember when they want to 
give up'/ 

Student : You just can't, we want to do good on our test and all 
and bring our points up and all. 

The passing through to the individual of the incentives facing the group 
can explain why cooperative learning does no worse than individual reward for 
individual learning, but it cannot explain why it does better. Three possi- 
ble explanations are as follows: 

• Group study motivated by a group reward is a more efficient 
method of learning than individual study . — 

• When rewards go to the group and groups are eyenly watched, every 
student now has a chance of being a winner, so students do not 
drop out of the competition ^ " 

• The total amount of rewards available for learning have in- 
creased in the group study environment . ~ 

Group study is more efficient . The interaction that occurs within a 
cooperative learning team can be thought of as reciprocal tutoring. The 
motivational dynamics are similar as well. In tutoring, each party can observe 
the effort and success of the other and, as a result, each is accountable to 
the other. In addition, the progress of the tutee is often being monitored, so 
the tutor's teaching success is often visible to others. Reviews of research 
on tutoring have found that it has large positive effects on the learning of 
both the tutee and the tutor. These positive findings suggest that the 
tutoring interaction induced by cooperative learning may be one of the reasons 
for its pos'Mve effects on learning. The failure of group work alone to have 
a positive effect on learning, means that a cooperative reward structure is 
absolutely essential for students to really commit their energy to the tutoring 
interaction. 

The interviews revealed that cooperative learning does induce students to 
take responsibility for how much their team mates learn. The following conver- 
sation illustrates how many of the students felt: 



Interviewer : What do you feel are your committments or responsibilities 
as a group person? 

Student : Sometimes when they don't understand, I try to explain because 
if its going to hurt them its going to hurt the team. 

Interviewer : How do you give each ^ther encouragement and praise? 

Student : I receive praise every day. It makes you feel good and want to 
do better. 

Interviewer : Are you being as successful in your other classes? 
Student : No, I am in English. 

Interviewer : What do you think makes the difference? 
Student : The team. 

Interviewer : Have you always been successful in math? 

Student : I can work with numbers real easy. Because I like uath a™j «ath 
relates to science. 

Interviewer : What do you do when your team mates don't do well? 

Student : I get upset, if I don't do well than I have no reason to get 
upset but if I do well. Then I know I haven't taught them. Or they are 
not listening. 

Another reason why team learning may be more efficient than traditional 
learning is that students find studying to be more fun because it is now a 
social undertaking. One female student in a cooperative class told us the 
fol lowing: 

Student : I am not the kind of person who can sit down for a long 
time. I must talk and socialize. The cl^ss is very casual. 
Learning is fun in team class ... You don't feel as if you're 
learning in class. 

Interviewer : Do you feel that team learning is a better way to 
learn? "~ 

Student : Yes it is because its funner (sic). Its something to 
look forward to at the end of the day. It makes you want to 
learn. 

A male student said the following: 

Student : I think cooperative learning has helped me a lot . . . 
Plus it's the teacher and how the teacher teaches. How much fun 
you put into the class. 
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Interviewer : Why this way of learning? 

Student : Because you have kids helping kids plus this way makes it 
more interesting. 

Every student has a chance of being rewarded . In cooperative learning, 
this characteristic is a function of the fact that competition is between 
evenly balanced teans. ILE scoring is often a part of cooperative learning, 
but with evenly balanced teams, it may not be necessary. Evidence that the 
"everyone-can-be-a-winner M effect is part of the reason for cooperaive learn- 
ing's success is provided by the four experiments discussed in section 1.4 
where rewards based on ILEs were implemented in classes that rewarded individ- 
ual learning. Important positive effects were found. This suggests that 
cooperative learning would probably not work as well if the teams were not 
evenly matched or if ILE scoring was not present. 

Total rewards have increased . Cooperative learning is a supplementary 
classroom reward system. Students continue to receive grades based on 
individual performance. Probably one of the important effects of cooperative 
learning is that it significantly increases the total amount of recognition 
given to student learning. Newsletters and certificates recognizing team 
accomplishments are always a part of a cooperative learning experiment but are 
seldom a part of the control environment. Brophy (1981) has observed, 
"classroom-process data indicate that teacher's verbal praise cannot be equated 
with reinforcement. Typically such praise is used infrequently, without 
contingency, specificity or credibility" (p. 5). Cooperative learning may 
change the way teachers use verbal praise but even if it doesn't, it 
significantly increases the amount of attention that is given to learning 
success. These effects of cooperative learning on the total amount of 
recognition being given students is illustrated by the following conversation. 

Interviewer : Before your were in the cooperative class, were you 
rewarded for working hard? 

Student : No. 

Interviewer : Were you embarrassed when you first went into his 
class and started receiving praise? 



ERIC 



Student : No, I liked it. (laughter) 



Interviewer : Did you? 

Student : Yes, I did. We had the thing before school starts. He 
said this is not going to be a very good class. This is going to 
be cooperative learning. You guy's are going to earn lollipops 
and I was so shocked. I said this is a math class. Crazy! 
(laughter!!) 

Interviewer : Other than lollipops, what kind of rewards did you 
geF? 

Student : Well, just the whole class claps and it just makes you 
reel good. Everybody has a smile. The whole class knows that you 
did good. 

Interviewer : Can you think back in your other classes and recall 
how your teachers reinforced your good grades? 

St udent : Well, last year nothing. I di 1't like my last year's 
"cTass at all. 

Interviewer : What were some of the differences? 

Student: It was boring. You just sat there and did your work and 
handed it in. Did your homework at night. 

Interviewer : Do you feel that team learning is a better way to 
learn? 

Student : Yes. 
Interviewer : Why? 

Student : It's pretty obvious, it's just funner. More fun to 
learn. I kinda look forward to it. Not only because it's the last 
period of the day, but it kinda makes you want to learn. Instead 
of (Oh! so boring I hate math!) I have to admit I hate math, but 
now it's not that bad. He puts it in a fun way. 
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CHAPTER 2 



A FORMAL MODEL OF SCHOOL REWARD SYSTEMS 
by Suk Kang 



In this chapter we present a simple mathematical model that describes how much 
effort students will put into preparing for an examination. The model offers a 
framework for answering the following questions: 

• Do students work harder if it is harder for them to pass the 
examination? 

• What types of students drop out when the passing standard is 
raised? 

• How does a pass-fail reward system compare to a graduated "eward 
system? 

• Is it possible to induce students to work harder by introducing a 
handicap system? If so, how should handicaps be determined? 



2.1 Pass-Fail Reward Systems 

Suppose students are given a qualifying examination in which the outcome 
of the test is determined in the following manner: if the student's test score 
is above a certain critical value, he or she receives credit (passes); however, 
if he or she fails, no credit is given. The test score depends on the stu- 
dent's effort and ability and on other random factors. The randomness of the 
outcome lies in the fact that (1) the test is an imperfect measure of knowledge 
and (2) the students are unable to predict exactly how much they will learn if 
they put a certain amount of effort into the task. Students value the credit 
because it improves their future prospects and because they receive immediate 
praise and approval from their parents and teachers. However, they may not 
enjoy preparing for examinations. 

Throughout this section, it is assumed that students choose their level 
of effort by considering the net expected benefit from passing the exam ana ex- 
pected effort (cost) required to pass it. Also, the students are assumed to 
be risk neutral, that is, their evaluation of outcomes depend on the expected 
value of net benefit. We assume that the test score is determined by the 
following relationship: 
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(!) T i ■ Tio ♦ Ei + Ui 

where Ti i s the i-th student's test score, Tio is the initial ability defined 
as the expected value of i-th students' test score if he or she doesn't put any 
effort into preparing for the exam, Ei is the effort level, and Ui is the ran- 
dom factor with variance o2. The probability of passing the exam is then 
written as follows: 

(2) Pr(Ti > TP) = Pr(Tio ♦ Ei ♦ Ui > TP) 

= Pr (Tio ♦ Ei - TP > - Ui) 

= F (Tio ♦ Ei - TP), 

where TP is the passing score and F is the cumulative density function of the 
random variable -Ui- 

Denoting the benefit of passing the exam by W and the cost of effort by 
C(Ei). the objective function of student i is written as 

(3) m^x W x Pr(Pass) - C(E n -) 
or 

(4) m|x W x F(T io ♦ E i - TP) - C(Ei) 

The solution to the preceding optimization problem depends on the values 
of W, Tio. Tp . and the shape of the cost function, C. In particular, the 
distribution function and the cost function play crucial roles in determining 
the response pattern of effort to the change in critical value. In what fol- 
lows we derive optimal effort level by specifying the distribution function and 
the cost function. 

Assuming that the random factor follows the normal distribution with a 
mean of zero and variance of o2, the first-order condition for the optimiza- 
tion problem is written as: 

(5) U x ff(T iQ - TP ♦ E )/o i )/o i - C'(E ) - 0, 
and the second-order condition is as follows: 
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(6) W x f'( (T iQ - TP * E^/o.J/o*- C"(E.) < 0, 

where f is tne probability density function of the standard normal 
distribution. 

In general, there are multiple values of Ei that satisfy the conditions 
for a local maximum, and the optimal level of effort is not a continuous func- 
tion of TP, More specifically, when the critical value is low, students tend 
to work harder as teachers raise the passing standard; however when the passing 
standard becomes too high (the exam is too difficult) , students tend to lose 
motiviation to work hard because they feel that the effort required to pass the 
exam is too high compared with what they receive for passing the exam. In the 
latter case, the student will decide to M take it easy" rather than to put more 
effort into preparing for the exam. 

We show a numerical example that exhibits the pattern described above, and 

derive several implications from the example. Let the cost function be <a 
constant elasticity type with increasing marginal costs: 

C(E) = E p , a > 1 
The first-order condition is given by the following equation: 

W x f ^ E i * T io " TP ) /o i) /o i - BE?" 1 x MR ( E i } " hL ( E i) = 0 

where MR(E.) = W x f((E. + T io - TP)/o i )/o i and MCfE^ = BE**" 1 

Figure 1 depicts the point at which the first-order condition is satisfied 
for a particular value of the passing standard. In figure 1 the MR curve is 
drawn for TP = Ti 0 . in this case the MR curve is the standard normal distri- 
bution function multiplied by W/oi and centered around zero. 
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*ure_l Deten.1n*tion of Effort Ltvel 



If T io - TP = 0, the optimal effort level is given by Ei at which MC(Ei) = 
hR(Ei). As the passing standard increases the MR curve shifts to the right, 
and the point of intersection also moves to the right. 

However, at some point, as the MR curve shifts farther to the right, it 
intersects the MC Curve three times. Figure 2 depicts such a case. 




If Ef* effort Itvtl 

Hqurt 2 Dtttnrinitlon of Effort Ltvtl 

At Ei = e x * and El = El **^ both the fl - rst . and tne S econd-order conditions 
for the local optimum are satisfied. The optimum effort level is determined by 
the comparison of the areas A and B. If the area of B is larger than the area 
of A, the optimum effort level is El**; however, if area A is larger than area 
B, the optimum effort level is E x *. Thus, the probability of passing the exam 
drastically changes when the inequality between the two areas reverses. When 
Ei** is the optimal choice (area A < area B), the probability of passing the 
exam is far greater than 50 percent but when El* is the optimal choice (area A 
> area B) the probability of passing is very low. In effect, the students 
have dropped out of the competition. At the point where the area of A coin- 
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cides with the area of B, there is a discontinuity in the reaction of student 
effort to an increase in the passing standard. Below the discontinuity, in- 
creases in the passing standard tend to stimulate effort. At the discontin- 
uity, an increase in the passing standard causes effort to drop to zero. In 
other words, the student drops out from competition in spirit, if not in body. 

Figure 3 depicts the relationship between the passing standard and the 
effort level. Note that as the student's effort level changes along the curve 
in figure 3, the probability of passing the exam also changes. 




Figure 3 Reaction Function 
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Table 2 shows the change in the student's effort level and the correspond- 
ing probability of passing in response to an increase in the passing standard. 
The numerical values in table 2 are calculated by setting 01 = o.5, W = 4.2, 
Ti 0 = 0. and 0 « 1.5. As the passing standard is raised, the student will 
increase his effort level. At a low level of the passing standard, the prob- 
ability of passing is very high (91X for a passing standard = 0.2), however, 
effort level is also low. The probability of passing decreases steadily and 
effort level increases as the passing standard rises. However, when the pass- 
ing standard becomes higher than 1.80, the student will stop trying to prepare 
for the exam, and the probability of passing becomes almost zero. 

The i-th student's effort level is maximized when the passing standard is 
1.80. Thus, if the teacher's objective is to induce students to extend their 
maximum effort, 1.80 is the optimal value of the passing standard. 

However, setting the passing standard at 1.80 may not be a wise choice. 
When there is uncertainty in the student's cost function, initial ability, 
and/or subjective assessment of the variance in outcome, the teacher will not 
know the exact value that represents the optimal passing standard. If the 
teacher sets the passing standard too high, the student will give up and dras- 
tically reduce his or her effort. If the teacher sets the passing standard 
somewhat below the 1.80, the student will reduce his or her effort only slight- 
ly. Thus, in the presence of uncertainty, it is desirable to set the passing 
standard somewhat below the point at which the student will decide to give up 
(for example, 1.8 in tf- (2 example in table 2). The degree to which the desir- 
able level of passing standard is lower than the point estimate depends on the 
accuracy of information on Tio and &• The more accurate the information, the 
smaller the divergence. 

Shifts in the Response Function 

The response function in figure 3 shifts when the parmeters representing 
the initial ability (Tio). the variance in test score (o?J, and the return 
from passing the exam (W) change. Figures 4A through 4C depict shift patterns 
of the reaction function when Tio is increased (4A), when o2. is decreased 
(4B) and when W is increased (4-C). In the figures, the curve 1-1-1-1 repre- 
sents the reaction of student 1 that corresponds to figure 3. 
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EFFCTT LEVEL AND PROBABILITY OF PASSING IN 
RESPONSE TO CHANGE IN PASSING STANDARD 



■ 0.5, U = 4.2, T 



10 



0.0, and e * 1-5, 



Passing Standard 
(Required test score) 



Effort Level (ex- 
pected test score) 



Probability 
(of passing) 



Expected Reward 



0.20 


0.87 


.910 


3.821 


0.40 


1.03 


.896 


3.764 


0.60 


1.20 


.885 


3.717 


0.80 


1.37 


.873 


3.666 


1.00 


1.55 


.864 


3.630 


1.20 


1.72 


.851 


3.573 


1.40 


1.90 


.841 


3.534 


1.60 


2.07 


.826 


3.471 


1.80 


2.25 


.816 


3.427 


2.00 


0.01 


.000 


0.000 


2.20 


0.01 


.000 


0.000 


2.40 


0.01 


.000 


0.000 


2.60 


0.01 


.000 


0.000 


2.80 


0.01 


.000 


0.000 


3.00 


0.01 


.000 


0.000 
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The T-T-T-7 curve in figure 4A is the reaction function of the person 
whose initial ability is higher than that of student 1. As the figure shows, 
those with higher initial ability put forth less effort for a given passing 
standard. On the other hand, they are also less likely to withdraw from the 
competition. 

The effect of a change in the perceived importance of random factors in 
determining performance on the test is depicted in figure 4B. The S-S-S-S 
curve represents a student who believes o$ to be small (that is, who believes 
that the test is reliable). The 1-1-1-1 curve represents a student who be- 
lieves to be large. When it is pt.xeived that passing the test is deter- 
mined by chance rather than effort (that is, is large), the payoff for 
effort goes down even though the impact of effort on learning has not changed. 
Students are mor likely to drop out of the competition, and those who remain 
in the competition will typically put less effort into it. Only when the 
passing standard is very low will an increase in the randomness of the test 
stimulate effort. The model implies that effort can be enhanced and dropouts 
forestalled by using tests that are more reliable and that are more responsive 
to effort and by enhancing the student's sense of self-efficacy (that is, the 
belief that effort rather than luck determines outcomes). 

Another factor that influences the student's reaction is the magnitude of 
the reward (W). When the reward for passing the examination is large, the stu- 
dents will put forth more effort. Figure 4C shows such a relationship. The 
curve W-W-W-W corresponds to the student W who expects a the large: reward for 
passing the exam than is student 1. The increased reward reduces the proba- 
bility that students will drop out of the competition and, at all levels of the 
passing standard, student W puts forth more effort than student 1. 

In summary, students increase their effort when rewards for passing a test 
are raised and more reliable tests are used. Reductions in the cost of learn- 
ing and effort (because of improved teaching or parental encouragement) also 
increase effort. Raising the passing standard increases effort up to a point. 
Beyond that point, the student stops trying, and drops out of the competition 
because the effort required to pass the test is more costly that the expected 
reward. 




Figure 4-A 



Change 1n Reaction Due to Difference In 
Initial Ability 




passing standard 



Figure 4-B Change In Reaction Due to Difference In 

the Variance of Test Score 




1 passing standard 



Figure 4-C Change In Reaction Due to Difference In 
the Reward 




passing standard 
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2.3 Graduated Rewards 

An alternative approach to rewarding students is that of distributing 
graduated rewards for improvements in performance. In the graduated reward 
system the reward is determined by P x (Ti - Tio). where P is the reward per 
point on the test and Ti is the i-th student's test score. The student will 
choose his or her effort by equating the marginal cost of effort to the mar- 
ginal return from additional effort. The objective function is written as 

max [P x (Ei + Ui) - E?]- 
The optimal effort level is given by 

E 1 = (P/e) 17 ^ 1 ) 

and the expected value cf payment is 

P x [(P/B) 1/(e " 1} ]. 

How well does a graduate reward system stack up against a pass-fail re- 
ward system? The efficiency of the twc systems will be evaluated by comparing 
the expected reward required to induce students to put forth a certain effort 
level in each system. If one system induces more effort than the other at 
given levels of expected reward, it is more efficient. The expected ^ost in 
the pass -fail system is obtained by multipling W by the probability jf passing. 
The expected cost in the graduated reward ystem is obtained by multiplying the 
expected value of the test score gain (Ti - Tio) by the reward per test point 
(P). 

The cost efficiency of the pass-fail system is obtained in the following 
manner: Given the error variance of the test, the optimal strategy in the 
pass-fail system is to choose bo** the size of the reward and the level of 
passing standard so that effort maximized at each level of expected expendi- 
ture. By changing the size of the reward and tracing the passing standard that 
maximizes the student's effort level, the corresponding expected resource 
requirement is obtained. The efficiency of the two systems is compared for 
each level of the expected resource requirement. Figure 5 shows the relation- 
ships between effort and resource requirement in the two systems. For the 
pass-fail system, the curves are drawn for three levels of the error variance 
of the test. 




•ffort level 
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The motivational impact of a pass-fail test is greatest when all students 
start with the same level of knowledge and skill and learn at the same rate. 
Criterion-referenced tests of new material more closely approximate this situa- 
tion than do competency tests for graduation. Since, however, background 
knowledge is often essential to learning new material and learning progresses 
at different rates in different people, there is no way out of the dilemma when 
tests are graded on a pass-fail basis. When one sets the passing standard, 
some students will find the standard so easy to achieve that they will not put 
out mere than minimum effort (if that is all that motivates them) and other 
students will find is so hard they will probably give up trying. 
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CHAPTER 3 



THE DESIGN OF HANDICAPPING SYSTEMS FOR REWARDING LEARNING: 
A MATHEMATICAL ANALYSIS OF INTERTEMPORAL ISSUES 
by John Bishop and Charles Wilson 

The primary objective of school recognition of learning is to induce stu- 
dents to push themselves closer to their potential. Schools cannot, however, 
reward everybody for everything. There is a limit to how much recognition a 
school can hand out, and handing out too many awards may reduce the perceived 
value of the award. If recognition goes only to those who score at the top of 
an absolute scale of knowledge and skill, most students will have no realistic 
chance of receiving significant rewards. This often results in these students 1 
giving up on the task of learning and disrupting the learning efforts of oth- 
ers. A reward system that gives everyone a chance at recognition and rewards 
only the learning that occurs because of extra effort rather than natural abil- 
ity is likely to be a more effective stimulator of learning than is general re- 
cognition of all learning. Since effort cannot be measured, the system cannot 
reward for effort. Learning can be measured, but what is needed is a handi- 
capping system that gives the slower students a more equal chance of being 
recognized for academic achievement. Unfortunately, there are difficulties in 
finding an administratively feasible way of measuring how much a student would 
learn in the absence of the reward system. The key to constructing an effec- 
tive system of recognizing academic achievement, therefore, hinges on finding 
an efficient way of defining a base score above which recognition is given. 

3.1 Alternative Reward Systems 

We will study systems of recognizing student learning with three contrast- 
ing ways of defining the base score of achievement above which graduated re- 
wards are offered: a fixed base score, a monthly updated base score reflecting 
previous month's achievement, and a base score computed on the student's peak 
performance over all previous months. 

The simplest program fixes a base score in period 0 at some level, L, and 
lets it grow (or decline) in each period by a constant factor, q. Although 
such a policy may be effective on a temporary basis, it is not necessarily 
desirable over a period of many months. A student's performance on a test at 
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the beginning of a school year is probably a reasonable predictor of perfor- 
mance during the first semester. However, it may be a poor predictor of his 
or her performance at the end of the school year or in the following year. 

The fixed base score policy will serve as a useful benchmark for evaluat- 
ing the efficiency of various alternative programs. The primary virtue of such 
a recognition system is that the actions of individual students in one period 
do not affect the recognition that the student can collect in the following 
periods. 

One commonly proposed recognition system is that of rewarding progress cr 
improvements over the student's performance on a previous test. The soecific 
system that will be evaluated t)ffers graduated rew* nis for raising one's score 
on a test above one's performance on the previous test. Declines in the score 
are not penalized. One might also wish to incorporate a growth or depreciation 
factor into the update rule* so that the threshold in each period is merely 
some multiple : last period's test score. In this case* the base score in 
each period t, Tt, is defined by Lt * pLt-1, where p is some positive constant, 
and Lt-1 is test score in the previous period. This type of recognition tends 
to be inefficient. Since a reduction in the test score on owe test .ewers the 
base score in the next, students my have an incentive to adupt a two-period 
cycling strategy in which they alternatively increase and then decrease their 
effort Sevel, thereby collecting the reward every other test. This tends to 
generate a relatively large reward payment, but very little net increase in 
learning. 

An alternative method of updating the base score that eliminates any in- 
centive to decr£*se Is* r^i^j is that cf jsiftj the st^v^n^s ^igh^s* test score 
since the beginning of the school year. A slight generalization of this idea 
is to define the baseline by Tt « q max(Lt-l # Lt-l)* **>ene q is a positive 
constant. If test score exceeds the base score in any period, the base score 
is automatically raised for all future tests. If test scores stay below the 
current base score, then the future base scores are unaffected. In this case, 
it simply increases (or declines) by a factor of q. Setting q * 1 yields a 
pi're peak system of recognition. Real-life examples of this reward system are 
found in track and field and in swimming where improvements in one's best time 
a.*e an important motivator for the athlete. 



3.2 The Model of Behavior 



We use a standard intertemporal model of individual beha/ior to analyze 
the response of students to the various types of reward systems. The student 
is assumed to choose an effort level in each period that maximizes the dis- 
counted sum of utility (rewards minus costs of effort) over an infinite plan- 
ning horizon. 1 The cost of effort in any period is a quadratic function of 
learning with no costs of adjustment. The assumption of no costs of adjustment 
and no interdependence of the learning and cost functions over time means that 
how much a student can learn in the nth period does not depend on how much he 
or she learned in previous periods. 2 Another crucial assumption is that, 
from the student perspective, the relationship between effort and test score 
does not have any random elements--f or example, the test meo^jres learning 
without error and the student can correctly predict how effort will translate 
into a tesfc-SAre (this assumption is relaxed in section 3.6) . We also assume 
that, in the absence of any reward system, the student's test score will in- 
crease or decline in each period by a constant factor g, which is perfectly 
anticipated. A formal development of the model is presented in appendix 3-A. 

The purpose of the model is to calculate the partial equilibrium responses 
of the student to the introduction of a permanent school reward system which 
rewards achievement over an individualized baseline. Therefore, we assume that 
other aspects of the environment such as the quality of teaching are unaffected 
by the reward system. 

3.3 Measures of Cost-Effectiveness 

Besides analyzing their impact on the student's behavior, we will also 
compare the effectiveness of these three alternative systems. As one might ex- 
pect, the marginal costs of improving one's test score increase with the size 

infinite planning horizons are assumed in order to examine the properties 
of handicapping systems when the system is in place for many years in a row. 
This creates a real challenge for a handicapping system because the student's 
individualized goal must carry over from 1 year to the next. 

^Learning independence across time periods might occur because (1) cummula- 
tive learning is occurring at the student's own pace or (2) the pace at which 
material is presented is fixed, learning new material dc^^ not depend on 
knowing old material and what is at issue is the degree of comprehension of 
the material presented. 



4i 43 



of the improvement that is desired. For a given reward structure, larger in- 
creases in learning will require more than proportional increases in rewards. 
Consequently, when comparisons across reward structures are made, rates of re- 
ward will be adjusted to produce equal increases in learning. The effective- 
ness of the different reward systems for specific students also depends on the 
rate at which the student would have learned in the absence of the reward sys- 
tem. Therefore, comparisons between programs will be made while holding the 
students underlying learning rate constant. The indicators of cost-effective- 
ness will be the cost to the school of the rewards offered divided by the extra 
learning that is induced. Since different reward systems venerate different 
flows of learning and reward costs over time, a discount rate is essential to 
evaluate learning gains over time. The discount rate, 6, which the student 
uses to discount future reward reflects the rate at which the student values 
future rewards relative to current rewards* If this also corresponds to the 
sociel rate of discount, then 6 is the appropriate factor for discounting both 
the subsidy and distortion costs. We will begin by assuming that society 
discounts future learning at the same high rate as does the student. In 
section 3.6, we will examine the effect of assuming that student* are much more 
present oriented than society, 

3*4 How Students Respond: A Comparison 

Before turning to an evaluation of the relative effectiveness of the three 
reward systems described in section 3.1 it will be useful to explore in some 
detail the differences in how students respond to these systems. Besides the 
rate at which learning is rewarded, the major determinant of the student's re- 
sponse to a reward system is the growth rate of the student's learning ability 
relative to the rate at which the base score depreciates. It is important, 
therefore, that we compare the students response to different reward systems 
for various exogenous growth rates of knowledge. To simplify the exposition, 
we will assume throughout that the base score depreciation rate for each system 
is zero. In this case, the updating rule for (1) the fixed base score system 
becomes Lt * Lt-l; (2) the updated base score system . It ■ Lt-l; and (3) the 
peak base score system , T t « max (Lt-l, U-l). 

A student's response to the different systems when his or her learning 
ability is growing extremely rapidly is Illustrated in figure 6. In each case, 

EF^TC 4? 50 
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O learning with no reward system 

• learning under fixed baseline reward system: I t « 1^ 

a learning under updated baseline reward system: I t ■ L t _j 

■ learning under peak score baseline system: T t - max(L t-1 , L^) 

Figure 6. Student response to different reward systems. 
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the student will respond by increasing learning by a fixed percentage in each 
period. The only difference in the systems is the magnitude of the response. 
Under a fixed base score system, the student's actual test scores have no ef- 
fect on the future base scores. Consequently, learning is increased by a fac- 
tor of (1 ♦ sn) where s is the proportionate increase in marginal rewards for 
learning resulting from the reward system and n is the elasticity of learning 
with respect to total rewards. Under the other two programs, however, increas- 
ing learning in any period also raises the base score for the next period. 
Therefore, the student will choose an effort level which equates the marginal 
gain from increasing learning today to the marginal loss from raising the ba*e 
score in the next period. As a result, learning is increased by a factor of 
only (1 + (1 - 6)sn), where 6 is the discount factor for future costs and re- 
wards. This means that, for a discount factor of .9 per month, a 10-fold in- 
crease in the reward rate is required to generate the same learning response 
under the monthly updated and peak period sy r t?tn as under the fixed base score 
programs. 

Inspection of the formulas in appendix 3-u reveals that as long as stu- 
dents follow a steady response to each reward system and the subsidy rates are 
adjusted to stimulate the same increase in learning, then the reward costs of 
each program are Identical. Although monthly updated and peak period systems 
require a higher rate of reward, the base score is continually updated so that 
a much smaller fraction of the test score 1s actually rewarded. Unfortunately, 
as the reward rate is increased, students require a higher growth rate before 
they will respond with the steady response illustrated in figure 6. For a 
reward rate of .4 and a discount factor of .9 per month, students with monthly 
updated base scores must have an exogenous learning growth rate in excess of 9 
percent if they are to adopt a steady response strategy. Learning ability 
clearly cannot grow at such a rate for long. 

For a range of lower growth rates, students facing the fixed or the peak 
score baselines will continue to adopt a steady response strategy. However, 
students facing the monthly updated baseline system will adopt a learning 
strategy in which the reward 1s collected once every two periods. In the first 
period, they will increase their learning by a factor of 1 ♦ sn; in the next 
period, they will decrease it by a factor of 1 - 6s". Then the cycle starts 
again. The result is illustrated 1n figure 7. Note that in those periods when 
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the reward is actually collected, the student responds exactly as if the base- 
line were fixed. This reflects the fact that since the student does not plan 
tc exceed the baseline in the next period anyway, the learning today cannot 
affect future rewards. In those periods when the threshold is not exceeded, 
however, the student does plan to collect the reward in the following period. 
Consequently, the student decreases his learning below the presubsidy optimum 
in order to lower the next period threshold. As a result, the gain in learning 
in the expanding periodr is at least partially offset by the learning decline 
in the contracting periods. As we might expect, the net result is a relatively 
large increase in both the distortion and reward costs. 

For growth rates near zero, the learning of students facing a peak score 
baseline will also begin to cycle, as illustrated in figure 8, After the stu- 
dent initially increases his or her learning to collect the reward, the thresh- 
old in the foUowing period may be sufficiently nigh that the student chooses 
to remain at his or her prereward system learning effort. As the ability of 
the student continues to grow, however, he or she will eventually choose to 
raise learning beyond the threshold established in the initial period and 
collect the subsidy again, thus starting a new cycle. If the students plan to 
wait n periods before collecting the subsidy again, the optimal learning in 
every "-th period is increased by a factor of (1 - «")s" + 1. In all other 
periods, it remains unchanged. The length of the cycle increases as the growth 
rate of the student's ability decreases and it goes to infinity as the growth 
rate goes to zero. For most parameters, however, the range of growth rates for 
which any cycling occurs is relatively small. For a subsidy rate of .4 and 
discount factor of .9 per month, cycling occurs only for growth rates between 
0 and 1 percent. In figure 8, a three period cycle is illustrated. 

Note that even in the periods when the student collects a reward, the re- 
sponse is less than it would be under a fixed baseline reward system. This is 
because any increase in learning today that exceeds the baseline imposes a cost 
in the form of a higher baseline in the future. However, as we should expect, 
the longer the student plans to wait to collect the reward again, the lesser 
the cost and, hence, the greater the learning response. The important point 
to emphasize is that, in contrast to a monthly updated baseline system, the 
student never has an incentive to reduce learning below the prereward system 
level . 
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I" the expected growth rate of the student's test score is negative, stu- 
dents facing peak score baselines will collect the subsidy once at most. Stu- 
dents facing a fi/ed baseline may choose to collect the reward for one or more 
periods, but eventually they too will stop responding to the subsidy. Except 
for extremely large negative growth rates, the learning of students facing the 
monthly updated baseline will continue to exhibit the two period cycle, but if 
the growth is sufficiently negative, they may choose to begin the cycle by 
first decreasing learning. Since later learning is discounted, it turns out 
that whenever this strategy is adopted, the present value of the learning gain 
is negative. For a subsidy rate of .4 and a discount factor of .9, this is the 
result whenever the growth rate lies between -1 and -9 percent per months. 
T r cal learning responses under the different reward systems for growth rates 
i this range are illustrated in figure 9. 

Except for the fixed baseline reward system, the most efficient responses 
to these r award systems tend to occur at the higher growth rates. The effect 
of introducing a depreciation factor into the threshold updating rules is 
essentially equivalent to an increase in the growth rate of the student's 
learning ability. In the next section, we will examine the cost effectiveness 
of these systems with and without a baseline depreciation factor. 

3.5 Cost-Effectiveness of Different Reward Systems 

It is important to recognize that the cost per unit of stimulated learning 
will generally be sensitive to the magnitude of the learning increase. The 
greater the increase in learning, the greater is the cost per unit of learning. 
In order to obtain a more appropriate comparison of the different programs, 
therefore, we will adjust che rate of reward for each program so that presently 
discounted learning rises by 1 percent at growth rates for which firms respond 
with a steady increase in employment (see figure 6). In general, this will 
imply different rates of reward for different programs. 1 

^"3 

The relative cost-effectiveness of the different reward systems may be 

sensitive to the level at which the baseline is set at the beginning of the 

program. For instance, if the initial baseline is set very low, the fixed 

baseline progiai? is essentially reduced to a nonmarginal reward of learning. 

Clearly, raising the baseline in this case will result in a dramatic reduction 

in the reward cosis. For the other systems, an increase in the initial 

baseline will also iower the reward costs; however, the decrease will not be as 

significant. If the initial threshold is set too high, however, students may 

not respond at all or, in some cases may even respond by initially reducing 
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Fixed baseline: L t = qLt-1 . As long as the growth rate of the baseline 
does not exceed that of the student's ability, a student will respond to a 
fixed baseline reward system exactly as ho or she would to a nonmarginal reward 
system. For a reward rato of s, the student will increase learning in each 
period by a factor of (1 + so), where n is the elasticity of demand for learn- 
ing. The distortion costs are also the same. The major difference in the two 
programs is in the costs of the reward system. 

Let us define the reward learning ratio of any program as the reward cost 
divided by the increase in the students learning. Then, using the formulas 
reported in appendix 3-B, we find that a nonmarginal reward system generates a 
reward-learning ratio of (s +1) whereas a fixed baseline system that grows at 
the same rate as the student's test performance generates a reward learning 
ratio of s. This means that for an elasticity of learning demand equal to .25 
and a reward cost of .04, the reward costs generated by a nonmarginal reward 
system are at least 4 times the amount by which the learning increases, while 
the reward-learning ratio for the fixed baseline system is only .04. In both 
cases, learning is increased by 1 percent. If student test-taking ability 
grows faster than the baseline, the fixed b^reline system becomes less effi- 
cient. For a constant threshold, the learn 'ng ratio rises to .43 if student 
test-taking ability grows at 1 percent, 1.94 if it grows at 5 percent, and 3.67 
if it grows at 10 percent. Other comparisons are presented in table 3. The 
fixed baseline system is almost always more efficient than a non-marginal sub- 
sidy. The problem with the fi *d baseline system is that the individual rather 
quickly stops responding to the incentives of the reward system if test-taking 
ability is declining. 



their learning. It is important, therefore, that our calculations be based 
upon the appropriate initial baseline. Last periods test score is a reasonably 
good predictor of this periods test score, so we will assume that schools would 
base the period 0 baseline for each student on their test score in period -1. 
In implementing our updating formulas, therefore, we will equate the baseline 
in period 0 with the presubsidy optimal learning for that period. 
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TABLE 3 

REWARD-LEARNING RATIOS* 
(For g = 10%, learning Increases by 1%) 



Exogenous 
Growth 
Rates 
(o) 



, No. 
Baseline 

(It., ■ 0) 



Fixed Baseline 



immediately 
updated Baseline 



Peak Score Baseline 



Prob of cont. 
wlOOl 



Prob of cont. 







q-1.0 


q«.9S 


p«1.0 


P-.95 


P-.90 


<ri 


q».9b 


q-1 


q«.95 


101 


4.04 


3.67 


3.80 


3.67 


3.80 


3.88 


3.67 


3.80 


1.13 


1.S4 


51 


4.04 


1.94 


2.67 


5.89 c 


2.67 


3.05 


1.94 


2.67 


.60 


1.07 


11 


4.04 


.44 


1.68 


3.89° 


3.05 c 


2.37 


.44 


1.68 


.13 


.68 


.51 


4.04 


.24 


1.55 


3.68 c 


2.91 c 


2.25 


.40* 
.40 # 


1.55 


.18 b 


.63 


01 


4.04 


.04 


1.42 


3.48* 


2.77° 


2.15 


1.42 


.40 # 


.57 


•11 


4.04 




1.15 


neg c 


2.51 c 


1.96 


.40 # 


1.15 


.40 # 


.46 


•51 


4.01 


mm 


.04 


neg e 


I.ST 6 


1.15 




.40* 




.40 # 


-101 


4.04 




mm 


mm 


neg c 


.87* 




mm 


mm 


mm 


Subsidy 
Rate 


.04 


.04 


.04 


.40 


.276 


.211 


.40 


.276 


.123 


.111 



♦These results are based on a discount factor 6 - 
"No response. 
*1 period cycle. 
b 4 period cycle. 
c 2 period cycle 

nes present value of learning Increase Is negative. 



.9 and elasticity of learning demand n ■ .25. 
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Updated baselines: Lt = p Lt-1 . We turn now to an evaluation of reward 
systems which use test performance in the previous periods to establish a 
baseline. As we illustrated in section 3.3, student response depends on the 
growth rate of exogenous learning relative to the baseline depreciation factor. 
Consider first the case where the baseline depreciation factor, p, is equal to 
1. Assuming a discount factor of .9, and elasticity of learning demand equal 
to .25, and a 40 percent reward rate, a students ability to perform on the test 
would have to grow by more than 9.8 percent per month for the firm to steady 
response Strategy. This is very improbable. 

Students with growth rates between -9 percent and 9.8 percent will respond 
with a two-period cycle. If the growth rate exceeds -1 percent the student 
will begin by increasing learning. In this case, the net increase in the pre- 
sent discounted value of learning will be positive but generally smaller than 1 
percent. In spite of this, the reward-learning ratio rises dramatically rela- - 
tive to the fixed baseline system. When the student lest taking ability dcss 
not change, the reward learning ratio is 3.48. Inspection of tables 1 and 2 
reveal similar results for other growth rates. For growth rates between -9 
percent and -1 percent, the firm will also adopt a cycling strategy; however, 
in these cases, the cycle will begin by decreasing learning. As we noted in 
section 3.3, this leads to a reduction in present discounted learning. 

The performance of the updated baseline system can be improved by setting 
the baseline equal to a fraction of last period's test score. For instance, by 
setting p * .95, students whose ability is growing at a rate of 4.1 percent or 
higher will adopt a steady response. When a steady response is generated^ costs 
and distortion costs become comparable to the fixed baseline system. For lower 
baseline depreciation factors, even more students adopt the steady response 
strategy, but ti 2 reward-learning ratio begins to rise as a greater fraction if 
the students 1 learning is rewarded. Some comparisons for a threshold deprecia- 
tion factor of .9 are also presented in table 3. 

Peak score baselines: Lt » q max (Lt-1. Lt-1) . For a given baseline 
depreciation factor, the cost-effectiveness of a reward system can be improved 
by using the peak scores on all previous tests as Lie baseline. In this case, 

the baseline updating rule can be written Lt * max(Lt-l, Lt-1). The gain from 
using the peak score updating rule over the immediate updating rule comes from 
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two sources: (1) many more students respond with a steady increase in learn- 
ing and (2) the response of those students who adopt a cycling response is more 
stable. Under the peak score baseline system, students never have an incentive 
to reduce learning below the level that is optimal in the absence of a reward 
system. 

Assume, as before, a discount factor of .9, an elasticity of learning de- 
mand equal to .25, and a 40 percent reward rate. Then, students whose test 
scores grow by more than 1 percent per period will adopt a steady response 
strategy, increasing learning by 1 percent in each period. 

Students with growth rates between 0 percent and 1 percent per period will 
adopt the cycling strategy illustrated in figure 8, For instance, at a g of 
one-half percent per period, they will increase learning once every 11 per- 
iods, generating a .98 percent increase in present discounted learning with a 
reward learning ratio of .40. Both of these numbers are higher than they would 
be with fixed baseline, but they are also considerably lower than under the 
updated baseline. 

One of the drawbacks of a pure peak score baseline is that only students 
with positive growth rates of learning ability will respond after the first 
period. One way to reach more students is to let the baseline depreciate each 
period by a constant factor q. This has the additional benefit of inducing 
more students to adopt the steady response strategy and also of generating 
greater learning from each student. For instance, if q = .95, the reward rate 
required to induce students following the steady response strategy to increase 
learning by 1 percent is lowered to .28. Furthermore, the steady response will 
now be adopted by students whose learning capacity is not declining by more 
than 4 percent per year. Unfortunately, lowering the baseline depreciation 
factor tends to increase the reward-learning ratio. In fact, the percentage 
increase is quite substantial for students with growth rates near 0 percent. 

Although this program is substantially more cost-effective than the up- 
dated baseline system, it is still the case that relatively large reward rates 
rre required to produce even a 1 percent increase in learning. This suggests 
that no f easible permanent program is likely to have much impact. However, as 
we shall see in the next section, when students are uncertain about the dura- 
tion of the reward system or students have much higher discount rates than 
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society, learning responses are larger and marginal reward systems are more 
effective. 

3.6 Uncertainty 

Thus far, the analysis has been concerned exclusively with the optimal 
learning response of students to various reward systems under the assumption 
that students do not face uncertainty and have the same discount factor as 
society as a whole. There are at least two ways in which uncertainty might be 
incorporated to the analysis. First, students may be uncertain about how long 
the reward system will remain in place or, equivalently may have a strong pre- 
ference for immediate rather than future rewards. This will not affect the 
behavior of the student under fixed baseline systems, but it dramatically im- 
proves the cost effectiveness of reward systems that use updating. The second 
kind of uncertainty that we might wish to examine is the student's uncertainty 
about his or her future learning function, or alternatively, the optimal amount 
of learning. Although we believe that this kind of uncertainty is important, a 
detailed analysis of behavior under these circumstances is beyond the scope of 
this chapter. We will confine ourselves, therefo r e, to a few brief remarks. 

High discount rates and uncertainty about the life of the reward system . 
If students have higher discount factors than society, the fact that getting a 
reward now will raise future baselines and make future rewards less probable 
becomes less of a concern. In addition, under almost any circumstances in 
which a reward system might be introduced, there is likely to be a significant 
probability that it will be discontinued at some point in the near future. 
Perhaps the simplest way to model this type of uncertainty is to assume that 
there is a fixed probability, (1 - w), that the reward system will be discon- 
tinued in any period—given that it was still in place in the previous period 
and that the student learns of the program's discontinuance at exactly the time 
at which it is discontinued. In terms of the student's objective function, an 
increase in this probability lowers the discount factor that the student uses 
to evaluate future rewards, while leaving society's discount factor unchanged. 

This does not affect the behavior of students facing a fixed baseline be- 
cause their current response does not depend on rewards expected in future per- 
iods. When there is updating, however, the student responds by trying harder. 
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This reflects the fact that future rewards are now less probable or valued 
less. 

In evaluating the learning gains and reward system costs, there is a dif- 
ference between lowering the society's discount factor and increasing the stu- 
dents subjectiv e probability that the reward system will be terminated. If the 
school actually intends for the program to be permanent, then the social gains 
and costs of the program should be evaluated at a discount rate which does not 
incorporate the possibility that the reward system will end. The net effect of 
an increase in either the student's subjective probability that the system will 
be discontinued or his or her discount rate, therefore, is to generate more 
learning at given rates of reward. These gains can be quite significant. 

Suppose a pure peak score baseline reward system (q = 1) is in place but 
that students believe that there is a 25 percent chance that the system will be 
terminated next period. Again assuming the discount factor is .9 and the elas- 
ticity of learning demand is .25, a subsidy rate of only 12 percent is required 
to induce students adopting the steady response strategy to increase learning 
by 1 percent. This is substantially smaller than the 40 percent rate required 
under the pure peak score update system with no uncertainty, but it is still 
three times higher than the 4 percent subsidy rate required under the fixed 
baseline system- The growth rates for which students adopt a cycling learning 
response still lie between 0 percent and 1 percent, and the distortion costs 
are also unchanged. The reward costs, however, are substantially lower. The 
reward learning ratio falls from .44 to .13 for students whose test performance 
grows at 1 percent per period. 

If e depreciation factor of .95 is introduced into the baseline updating 
rule, the cost-effectiveness of the system for improving students is reduced; 
however, it still compares reasonably well with the fixed baseline system, even 
without a depreciation factor. Except for growth rates near 0 percent, the 
reward-learning ratio is still lower under the peak score updating system, and 
more students respond to the system. A systematic comparison of the cost ef- 
fectiveness of the peak updating under uncertainty with the other programs is 
presented in tables 2 and 3. These results suggest that peak score updating 
may be considerably more effective if students are even a little uncertain 
about the permanence of the reward system. 
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Uncertainty about future test scores . The basic principles that govern 
the student's response to a reward system under perfect certainty do not change 
when we introduce uncertainty about his or her future optimal learning. It re- 
mains the case that the student's response under a fixed threshold program de- 
pends only on the relation between increasing learning today and raising the 
baseline in the future. The difference is that the size of the future fca^eline 
relative to its future optimal learning level is now a random variable. 

Although there may, consequently, be more fluctuations in the learning re- 
sponse in this case than there would be under perfect certainty, we should 
still expect peak score updating system* to be more cost-effective than immedi- 
ate updating. The reason for this is the same as in the previous case. Im- 
mediately updated baselines sometimes produce an incentive to lower present 
learning below the optimal level in order to lower the baseline in the next 
period. Under a peak score baseline, the future baseline can only be affected 
if the baseline is exceeded in the current period. Since future rewards are 
discounted, the student will never choose to lower learning below the optimal 
level in the current period. We see no reason, therefore, why the presence of 
uncertainty should change our conclusion that a peak score baseline is a more 
cost-effective way of increasing learning than 1s immediate updating. 

Baselines that are averages of past scores . Updating individualized 
learning expectations by taking averages of past scores seems to be an attrac- 
tive way of dealing with uncertainty regarding learning rates and the fact that 
monthly tests measure true learning with some error. Such reward schemes were 
not subjected to a rigorous mathematical analysis. However, our examination of 
the simpler case of immediate updating leads us to conclude that updating base- 
lines with a running average of past performance has many of the same problems, 
though generally to a lesser degree. The student response to the subsidy is 
still reduced by the fact that future baselines will be higher and rewards 
harder to get. The greater average delay 1n the response of baselines to past 
performance means that the behavioral responses are greater for a given rate of 
subsidy. When there is no uncertainty about the effect of effort on learning 
rewards, updating based on a running average will reduce cycling. Test per- 
formance has a random component, however, and th.i, combined with the fact that 
the student can purposely do poorly on the test, reopens the possibility of 



cycling. Clearly, a lower bound must be placed on the scores that are included 
in the average which updates the threshold. 

3.7 Conclusion 

What may be concluded from the exercises we have reported in this paper? 
One of the most important conclusions is that it appears very difficult to 
devise reward systems which are more cost-effective than a simple marginal 
reward system with a fixed threshold in each period. If the optimal learning 
of the average student is growing (or declining) over time, it may be desirable 
to introduce a growth or depreciation factor into the program. Alternatively, 
the base ine of each student might be deviated from the average score of all 
students in the class. As long as the individual student cannot affect his or 
her baseline, however, the learning decision will not depend on how the base- 
line is updated. 

A fixed baseline system also generates the maximum learning gain for a 
given reward rate. For a discount factor of .9, the learning response is 10 
times higher when the baseline is fixed than under any other updating rule in 
the absence of uncertainty. Because there are no intertemporal trade-offs when 
the threshold is fixed, students respond to the subsidy as if there were a 
permanent increase in the payoff to learning. 

There is only one instance in which we have been able to demonstrate the 
possibility of a substantial improvement over the cost-effectiveness of fixed 
baseline reward systems. This occurs when students believe that there is some 
probability that the reward system will be terminated next period. Such a be- 
lief g'qps not change the student's behavior when the baseline is fixed. How- 
ever, the student responds differently in reward systems with a trade-off 
between more learning today arid a lower baseline tomorrow. Since the students 
cannot be certain of obtaining future rewards, they try harder in each period 
in order to be certain of obtaining the rewards while these are still avail- 
able. Thus, if students are sufficiently uncertain about the future of the 
reward system, it is possible to use past test scores to update the student's 
baseline to increa e the cost effectiveness of the reward system. 

If it is decided that a reward system should include a baseline updating 
rule based on past test scores of the individual, then it is important to de- 
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sign the program in a way that reduces the fluctuations in the learning re- 
sponse. Of the schemes we have considered, our analysis suggests that a system 
that adjusts the baseline to reflect the previous peak test scores will be most 
effective in meeting this objective. Unlike systems with immediate updating of 
thresholds, peak score updating schemes never provide an incentive to the stu- 
dent to reduce learning below the optimal level. If there is no depreciation 
of the baseline, however, only students with a positive learning growth rate 
will be rewarded more than once. By introducing a depreciation factor into the 
baseline updating rule, not only will more students respond to the subsidy, but 
the learning response of each student will be larger for the same subsidy 
rates. This effect tends to partially offset the higher reward-learning ratio 
resulting from the lower baseline. 

This initial foray into formal mathematical analysis of the incentive ef- 
fects of alternative classroom reward systems has produced some important in- 
sights. Further work will almost certainly yield more. Since running average 
updating rules are used in Slavin's cooperative learning programs (Team Assis- 
tant Instruction and Student Team Achievement Divisions), careful analysis of 
their properties is called for. In order to keep the problem tractable, we 
had to make a number of assumptions that do considerable violence to class- 
room reality. In our view, the most serious problems lie in the assumptions 
(1) that there is no error in measuring learning and (2) that the marginal 
costs of underlearning are comparable to the marginal costs of overlearning 
(that is, the learning cost function is quadratic). In fact, students can 
purposely do poorly on a test, therefore, reward systems that lower baselines 
substantially when a student does very poorly can be easily undermined by 
strategic behavior. Our conclusion is that reward systems should either not 
update their baselines or should ignore low scores when baselines are being 
updated (as the peak period updatir.y system does). 
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APPENDIX 3-A 
THE MODEL 



We assume that the objective of a typical student is to choose his or her 

learning level in each period to maximize the discounted sum of utility. In 

order to obtain simple closed-form solutions, we assume that ability in any 

period is a concave function of student learning in that period, Lt» given by 
a 2 

^Lt " ^1*) ^ " w ^ ere L| is a parameter proportional to the optimal 
learning in period t and a and b are positive constants. Let w be the cost of 
learning above baseline It. The the net utility in period t is equal to 

bL t m ^ l t ml V z - wL t + sw mdX < 0 ' L t- L t>- 

With no loss of generality, we may assume that b = w.l Then, if the student 
discounts utility in each period by a factor of 6, the problem of the student 
is to choose L in each period to maximize 

(i) t l>(^Hhf L t )2 + sw max (0 » V L t^ fit 

where It is typically a function of Lt-l and Lt-1- 

It is clear from inspection of (1) that in the absence of a reward, the 
optimal policy of the student is to set L£. Thus. L£ may be regarded as the 
optimal learning during time period t. The utility function has been con- 
structed so that the cost of a given percent deviation from the optimal 
learning level is proportional to that optimal level. To obtain steady-state 
solutions, we will assume that Lt grows each period by a constant 



llf b * w, let a' = (a*^w ^ l* 1 , {1 + ^) L *, and c t ■ L*(b-w)(a^ ). 

a 2 a 1 

Then pr(L t - L^) ♦ (b-w)L t = pfT(L^ - ♦ c t . Since c t is a constant, it 

can be ignored in determining the optimal response of the student. 
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factor g(0 < g < 1/6). We also assume that both the cost of learning and the 
utility function are unaffected by the reward system. 

By appropriately normalizing units and adjusting the value of a, we may 
set L^/g* = L*, = 1. Let x = g« < 1 be the discount factor normalized for 
growth and let * t be the student's learning level in period t, normalized for 
growth. Then the student's problem may be rewritten as 

("?£) tlo M*t" 1}2 * sw max(0 » *t"V gt)]Xt - 

In the absence of the reward system, the student will set *t = 1 in each 
period. 

Finally, we may show that at the no-reward optimal learning the elasticity 

of demand for learning, n, is equal to l/2a. Ignoring the reward, the single 

a 2 

period utility function for the student is (L-L*) ♦ (b-wlL. Differentiat- 
ing with respect to L and setting the resulting expression equal to zero gener- 
ates the optimal learning level, L = fejj ♦ 1)L*. Then n = ~- & « ♦ 1) . 

t a ^ w l~ a 

(^)- Then if D = w = 1, n = l/2a. 



AppendiA 3-B 
FORMULAS USED IN TEXT 



We present here the general formulas for the present value of the increase 
in learning and reward costs produced by the various reward systems discussed 
in the text. These formulas are based on the formal analysis of utility 
maximizing behavior presented in appendix C (Bishop-Wilson [1980]). The 
reward-learning ratios presented in table 1 are equal to the reward cost 
divided by the learning gain. Formulas for distortion ratios are presented but 
not used to create tables. 

1. Fixed Threshold Program: L t = (9 > 0)* 

Assume that I = qL* 
o 1 -1. 

(a) If g ^ q, the student sets * t = sn + 1 for all t 2 0. In this case, 
we have 

Learning gain: y^T 
Distortion costs: y-ry" 



Subsidy costs: s(^_^ - _fl4_, 



(b) If g < q, then there is a t such that t = max {t:=p < g t (l « ™)} . 

Then, * t = 1 ♦ sn if t < t; 1 1 » 0 for t > t. If p > g(l + -^J), then t < 0 and 
there is no response. Otherwise, we have 

1 - >T t+1 

Learning response: sn( ^ _ - — ) 

.2_ . t+1 
Distortion costs: i^]-^ — ) 

Subsidy costs: s[(sn ♦ 1) - \ ^j^ )] 
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2. Yearly Updated Threshold : L t * pL t _ 1 (p > 0). 

™~ 5 - : &)Vv = a ■ nVMS 8 )- »««- ^ r „ ■ n! r 

(a) If | > g * g, then the student sets * t = (1 - p*)sn ♦ 1. In this 
case, we have 

Learning gain: ( | ~ x & ) sn 

2 

Distortion costs: ^ %Z ^ Z 

2 

Subsidy costs: ^ ^l"-^ ) sn ♦ 1 - and 

(b) for g > g * 3, the student follows a two period cycle. Learning 

alternates between sn ♦ 1 and -pfisn ♦ 1. Define g = ^ » — . 

sn(l - (pt) £ ) ♦ 2 

(i) If g > g Q , then the student responds by first increasing learn- 
ing (i.e., £ Q « sn ♦ 1). In this case, we have 

Learning gain: ( 1 " fy^ Jsn 

1 - X 

2 2 

Distortion costs: { l * M ) V 1 

1 - X* ' 

Subsidy costs: s ^t 1 * ll&h-t 1 : EZfl) 

1 - X* 

(ii) If g < g Q , then the student responds by first decreasing 
learning (i.e., i Q = -pfisn ♦ 1). In this case, we have 

Learning gain: { (9 I Pl sn < g. 

1 - x' 

Distortion costs: M> 6 f t* ) 4n 

1 • X Z 

Subsidy costs: sAfsnl ♦ ( P 6) 2 ) ♦ 1 - (p/q)] 

1 - x z 

(c) For g < 5, the student does not respond. 
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3. Generalized Peak Learning Threshold : L t = q max (L t _ 1$ L^j) (q > 0). 

Assume that L Q = q! . As in the text, let n be the probability that the 
reward will not be discontinued in any given period, given that it has stayed 
in place up to that period. 

(a) If g > q t the student will respond by setting = sn(l - (*p6) n ) 
for t = 0 t n t 2n t ... and t = 1 in all other periods. The length of the cycle 
is determined by choosing an n that maximizes 



F (m) = i 1 ~ PH 6 ) )(l - p(irq6) m ) ^ ♦ 1), In this case, we have 
l - P(n) m 6 



Learning gain: t 1 " PH 6 )^ sn 

1 - X 

n 2 2 

Distortion costs: I 1 " PW) ) sn 

1 - a" 2 



Subsidy costs: s[( l " PW )(sn(l - p(irq6) n ) ♦ 1) - £3] 

1 - X 9 



(b) If 0 > g > 2 2 + sn » the student wi 1 1 respond with i Q = 1 + sn and t 
st^ = 1 for all t > 0. In this case, we have: 

Learning gain: sn 

2 

Distortion costs: 

Subsidy costs: sn ♦ 1 - « 

(c) If 0 * T+sn* *^ en tnere ^ s no re sponse. 
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APPENDIX A 
OBSERVATIONAL DATA COLLECTION 



The overall purpose of this study was to develop theoretical frameworks 
for understanding the influence of high schools on youths' adaptability to 
workplaces. In concert with the theorization activity, observational -based 
research addressed questions about instructional methods, contexts of learning, 
and peer influence in the classrooms. The unit of analysis in this research 
was the classroom. Our sampling scheme emphasized exemplary programs as op- 
posed to the standard classrooms; it emphasized variation in student motivation 
among members of the classes; and it emphasized variation in type of classes. 
This appendix provides a chronology of the observational research and docu- 
mentation of the resulting data. 

Sample Selection and Data Collection Procedures 

The observational data collection component of the study involved a pilot 
site and three exemplary programs. 

Pilot Site 

During spring 1985, many Columbus-area school districts were contacted and 
asked to serve as a pilot si'-e. Project staff met with district personnel in 
some cases and mailed information concerning the project in others. Since it 
was the end of the school year, many districts were reluctant to participate 
and oftentimes the district's policies and procedures would have taken longer 
than our time frame would allow. 

The Westerville City School District, a suburban district of Columbus, 
however, agreed to participate as the pilot or "test" site for this research. 
The Westerville administration was very responsive, and since there are two 
high schools in the district, we were able to use the entire project staff for 
observations without "overloading" one ,, r hool; also we were able to compare 
experiences across schools. 

The following is a de$' nption of Westerville North and Westerville South 
high schools: 
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• Westerville City School System 
Westerville North High School 
950 Smothers Road 

Mr. Lee Whitaker, Principal 

• Westerville South High School 
303 S. Otterbein Avenue 

Nr. Richard Miller, Principal 

South and North high schools are located in Westerville, a growing 
residential suburb of Columbus, Ohio. North opened in 1975 in 
response to the rapid growth of the community. 

Both schools are comprehensive high schools, but mainly emphasize 
academic programs. Independent and college placement programs are 
available to students on a contractual basis. Alternative (gifted 
and talented) and honors courses are limited in enrollment and 
offered to students selected on the basis of faculty recommenda- 
tions and test scores. Each school also offers strong extracurri- 
cular programs that include outstanding music and drama depart- 
ments, a wide variety of clubs and organizations, and highly suc- 
cessful athletic departments. 

Surveys indicate that 55-60 percent of the schools' graduating 
seniors plan to continue their education at 2- and 4-year institu- 
tions, 6-9 percent enter the military for further training, and 
many of the others go into the work force. 



• South High School (113 certified staff) 

School Enrollment: 1,945/982 boys; 963 girls 

North High School (93 certified staff) 

School Enrollment: 1,660/827 boys; 833 girls 

93 percent white; 7 percent minority 

Classroom observation occurred in 12 separate class periods--2 periods a 
day for 6 teachers--from Wednesday, May 8, through Monday, May 13. Two classes 
of English, Chemistry, and Social Studies were observed in each school. To 
test the methodological question of the appropriateness of 1 or 2 observers, 
50 percent of the class periods had 1 observer and 50 percent had 2. All in 
all, the pilot site involved 72 classroom hours of observation. After the 
observation period, each teacher was interviewed, a total of 5 students from 
each of the 12 class periods were interviewd, guidance personnel were inter- 
viewed, and the assistant principals for curriculum in each high school were 
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interviewed. Thus, the sample size for interviews in the pilot study was as 
follows: 



• Administrators . 

• Teachers . . . . 



2 



• Guidance counselors . 2 

• Students 60 

The interviewing took place from Monday, May 13, through Thursday, May 16. 

Following the pilot site observation, the observer/interviewers' notes 
were coded and typed. The project staff met and discussed the substance of 
their experiences and addressed various methodological issues to inform the 
observation process scheduled for fall. 

Exemplary Sites 

A number of sources of information were used to identify the exemplary 
classrooms. Teacher publications were reviewed for articles about such pro- 
grams. Representatives of teacher membership *nd other professional organi- 
zations were contacted. NIE staff and colleagues at other universities and 
labs and centers were contacted for their suggestions. Finally, nominations 
for exemplary programs in the area of education and employment that had been 
submitted to the National Center for a recent audioconference were reviewed. 

An explicit criterion for a school or classroom program to be categorized 
as exemplary included documented evidence on the effectiveness of the programs. 
We limited our search to nonvocational programs; to high schools with a sig- 
nificant number of college-bound students; large schools with at least 1,500 
students; and, in one case, to a school that used cooperative lear 7 tech- 
niques. 

Table A-l provides a list of individuals and organizations that were 
contacted for suggestions of exemplary programs. Table A-2 provides a list of 
the exemplary program suggestions received. Following are brief descriptions 
of the three exemplary sites that were chosen. 
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TABLE A-l 



LIST OF INDIVIDUALS AND ORGANIZATIONS CONTACTED 
FOR EXEMPLARY PROGRAM SUGGESTIONS 



Ms. Pat SchwalHe-Giddis 

State Coordirator of Career Education 

Tallahassee, FL 



Dr. Helen Hill 
Department of Education 
Richmond, VA 



Mr. William Parrish 

National Association of Secondary 

School Principals 
Res ton, VA 

Ms. Nona Denton 

Illinois Board of Education 

Springfield, IL 

Mr. Carl Larsen 

Iowa Central Community College 

Fort Dodge, I A 

Dr. Mary Anne Raywid 

Chair, Educational Administration 

Hofstra University 

Dr. Marion Holmes 

Executive Director of Career 

and Vocational Education 
Philadelphia, PA 

Dr. John Meerbac* 

Chief, Division of Student Services 
Harrisburg, PA 

Ms. Marian Craft 

Office of Vocational and Adult 

Education 
U.S. Department of Education 
Washington, DC 



Dr. Steve Hamilton 

Department of Human Development 

and Family Studies 
Cornell University 

Dr. Lanso A. Crim 
Superintendent 
Atlanta Publ ic Schools 
Atlanta, GA 

National Child Labor Commitcee 
New York, NY 

Dr. Thomas Owens 
Senior Research Associate 
Northwest Regional Education Lab 
Portland, OR 

American Association of School 

Administrators (AASA) 
Arlington , VA 

National Alliance of Black School 
Educators (NABSE) 

National Commission for Cooperative 

Education 
Boston, MA 

Cooperative Education Research Center 
Boston, MA 



Ms. Frances Cloyd 

Coordinator of Community Relations 

and Career Education 
Lexington, KY 

Dr. Richard Campbel 1 
Director 

Cooperative Education/Youth Employment 
Lincoln, NE 

Dr. Richard Miguel 

National Center for Research 

in Vocational Education 
Columbus, Ohio 

Professor Robert Slavin 
Johns Hopkins University 
Baltimore, Maryland 

EBCE Dissemination Project 
^ Berkeley, California ( 
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American Vocational Association 
Arlington, VA 

Council of the Great City 

Schools (CGCS) 
Washington, DC 

Dr. Ronald Bucknam 

National Institute of Education 

Washington, DC 

Mr. Don S. Ayers 
Supervisor, Secondary Guidance 
Programs 

Virginia Department of Education 

Association for Supervision and 
Curriculum Development (ASCD) 
Alexandria, VA 22314 
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TABLE A-2 
EXEMPLARY PROGRAM NOMINATIONS 



School 



Source of 
Recommendation 



Program bescnption 



New Garden Friends School 
Greensboro, N. Carolina 



Mary Anne Raywid 



Metro Secondary School 
Cedar Rapids, Iowa 



Linn-Mar High School 
Marion, IoWd« * 



J.F. Kennedy High School 
Bloomington, Minnesota 



Parkway North High 
Creve Coeur, Missouri 



Boise 70001 Project 
Boise, Idaho 



North Central High 
Indianapol is, Indiana 



Highland High School 
Salt Lake City, Utah 



ASCD 



ASCD 



ASCD 



ASCD 



Robert Gilbert 



Audioconference 
Nomination 



Audioconference 
Nomination 



This small private school was 
established on the principles 
of Friends belief in the worth 
of the individual. Students 
proceed at their own pace and 
are taught to be independent. 

This high school offers 

a special prooram in career- 

education/youch employment. 

This high school offers a 
special program in career- 
education/youth employment. 

This high school offers a 
special program in career- 
edu^ation/ycuth employment. 

J\t s high school offers a 
special program in career- 
education/youth employment. 

Boise 70001 is part of a 
national nonprofit corporation 
that helps 16 to 21-year-old 
high school dropouts. It is a 
program of employment, 
training, education, and 
motivation. 

"Learning Unlimited" offers a 
community-based career 
exploration program. Students 1 
work is individualized and 
contracted with both teachers 
and parents. 

This voluntary alternative EBCE 
program uses the entire 
community as a school. The 
program focuses on direct 
experience learning in a 
community setting where 
students apply classroom theory 
to solve practical problems. 
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Table A-2--Continued 



School 



source or 
Recommendation 



Program uescnption 



Waverly Public Schools 
Waverly, Nebraska 



Olney High School 
Philadelphia, r'A 



Tigard High School 
Tigard, Oregon 



Cincinnati School for 
the Creative and Per- 
forming Arts 
Cincinnati, Ohio 



Richard Campbell 
Nebraska Dept. of 
Education 



Tom Owens 



Tom Owens 



Kathy Borman 



Las Cruces Public Schools Tom Owens 
Las Cruces, New Mexico 



Southside Senior ».igh 
The Greenhouse School 
Rockville Center, New York 



Mary Anne Raywid 



Jefferson County Open H.S. Audioconference 
Evergreen, Colorado Nomination 



This program involves the 
teaching of employability 
skills, on-the-job training in 
specific occupations, integra- 
tion of handicapped, disadvan- 
taged and regular students 
into one program. 

The Academy for Career Edu- 
cation offers a comprehensive 
program responsive to students 
academic, personal, and 
vocational development. 

This high school offers an 
outstanding program in career 
exploration through community 
experience. 

SCPA provides an educational 
environment that makes the 
arts the center of student 
attention. All students mist 
audition before the school 
faculty for admittance. 

EXCEL-the career program is an 
experience-based career 
education (EBCE), designed to 
explore careers in the 
community while earning school 
credit. 



This is a small public 
alternative school with a 
Walkabout-based curriculum 
that is experiential, chal- 
lenging, ung-aded, and indivi- 
dual ized. 
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Table A-2--Cnntinued 



iicnoo i 



Source ot' 
Recommendation 



Program Description 



Mt. Hebron High School 
Ellicott City, MD 



The Village School 
Great Neck Schools 
Great Neck, New York 

Philadelphia Job Search 
John F. Kennedy Center 
Philadelphia, PA 



Murry Bergtraum H.S. 
for Business Careers 
New York. New York 



BOCES 

Yorktown Hts., NY 

Gloucester High School 
Gloucester, Virginia 



John Bishop 



Mary Anne Raywid 



Marion Holmes 



Committe for 
Economic Develop- 
ment Report 



Peter Copen 



Don S. Ayers 



Fayette Co. Public Schools Ron Bucknam 
Lexington, KY 



The math department has had 
extensive involvement with 
cooperative learning; it is now 
being introduced to other 
academic departments. 



This program operates in sev- 
eral high schools to bridge the 
gap between school and work by 
providing students with 
important skill needed to 
acquire a job now and 
throughout their lives, 

Bergtraum provides a strong 
business careers orientation; 
students are exposed to a 
variety of opportunities for 
both cultural enrichment and 
social involvement. 

Walkabout Program emphasizes 
a career preparation and a 9- 
week career internship. 

This school has an exemplary 
"youth employability program, 
the Gloucester-Mathews Job 
Referral Service, 

The Fayette County Schools 
offer an experienced-based 
career education program, 
operating in four high schools 
that combines "classroom" 
learning with "real world 1 ' 
experiences. 
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Table A-?--Continued 



Schoo l 



"Source or 
Recommendation 



Program Description 



Linworth Alternative H.S. 
of Worthington High 
Worthington, Ohio 



Richard Miguel 



Jefferson High School 
Portland, Oregon 



Audioconference 
Nomination 



In a highly competitive 
college prep high school the 
alternative high school 
provides students the option to 
plan their curriculum based on 
their particular learning 
needs. 

The Financial Services Academy 
is a unique industry-school 
partnership combining practical 
training related to financial 
services with basic academic 
studies. 
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(1) 



Mount Hebron High School 
9440 Route 99, Old Frederick Road 
Ell ;cott City, HD 21043 
James R. McCrumb, Principal 



Mt. Hebron was selected as an exemplary site for its extensive 
use of cooperative learning techniques, used largely in the math 
department. Mount Hebron is located outside of Ellicott City in 
central Maryland. The Mount Hebron faculty is composed of 47 full- 
time and 4 part-time teachers. In addition, Mount Hebron has three 
administrators, three guidance counselors, and two media special- 
ists. 

Mount Hebron is a comprehensive high school that includes grades 
9-12. The majo, ity of courses are academically oriented and in- 
clude advanced placement offerings in English, social studies 
(European and U.S. History), science (biology, chemistry and phy- 
sics), and French. The math department offers three levels of 
calculus (through differential equations) as well as computer 
programming (basic and pascal). In addition, all academic depart- 
ments have developed underclass honors and/or Gifted and Talented 
P ograms. Mount Hebron also has a well-developed business curri- 
culum within the school and a vocational program to which our 
students ere bussed. 

Each year approximately 75-80% of graduates continue their educa- 
tion at postsecondary institutions. Of the 220 members of the 
Class of 1985, 54.5 percent entered a 4-year college; 22 percent a 
2-year college; and 2 percent a trade, technical, business, or 
nursing school. Almost all of the rest were able to obtain entry- 
level jobs (17 percent) or enter the armed services (2.5 per- 
cent). 

School Enrollment: 1,004/489 boys; 515 girls 

4 percent Asian; 6 percent black; 90 percent 
white 

Observations at Mt. Hebron were centerd in th r math department, where 
cooperative learning techniques we<*e being used. A total of nine Masses were 
observed—three classes each for three algebra teachers (2 classes used cooper- 
ative learning anc? one not using cooperative learning). Additionally, five 
students from each class, the t;iree teachers, a guidance counselor, and the 
principal *>ere interviewed. Observations began on Wednesday, October 23d and 
were conducted fcr 4 consecutive school days. Interviews were conducted over 
the period Monc^y, October 28 through Thursday, October 31. 
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(2) Cincinnati School for the Creative 

and Performing Arts 
1310 Sycamore Street 
Cincinnati, OH 45210 
Mr. William Dickinson, Principal 



Cincinnati's innovative School for the Creative and Performing frts 
(SCPA) operates from a philosophy that combines elitism with a 
concern about the arts. The arts are viewed as a means of achiev- 
ing racial integration, with the means being as important as the 
end. Elitism is essential because the school recognizes and pro- 
motes the best talent through its requirement that children are 
only admitted after a successful audition before the school facul- 
ty and through its selection of only the best talent for public 
performances and recognition. The grade range in the school is 
from the 4th to the 12th grades, 

Though each student is involved in a college preparatory curricu- 
lum, all students are required to take at least one class in each 
major, arts area. The school defines these major arts as dance, 
drama, instrumental music, and visual arts. In addition, all 
students must major in one area of the arts such as creative writ- 
ing, dance, drama, instrumental music, music composition, vocal 
music, visual art, fine arts, music theater, and stagecraft. 

SCPA has an annual average of 74 percent of its graduates who 
pursue higher education at a 4 year institution and another 15 
percent continue at a 2-year or technical institution. 

School Enrollment: 1,120 students/467 boys; 653 girls 

45 percent black; 52 percent white; and 3 
other 

Classroom observations of two classes of Psychology, History, and Drama 
Lab were conducted from Wednesday, September 18th, through Monday, September 
23d at the Cincinnati School for the Creative and Performing Arts. During th 
week of September 23d through Septenber 30th, the principal, the teachers of 
the classes observed, and five students from each class were interv ewed. 

(3) Hurry Bergtraum High School 

411 Pearl Street 
New York, NY 10038 
Ms. Barbara Christen, Principal 

Murry Bergtraum High School for Business Careers is a coeducational 
secondary school located in the City Hall area of downtown New York 
City. Because of Bergtraum 1 s proximity to the Manhatton Civic 
Center, financial district, and institutions of higher education, 
students are exposed to an extraordinary variety of opportunities 
for both cultural enrichment anc* social involvement. 
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Because of Murry Bergtraum' s strong business careers orientation, 
the school attracts in excess of 25,000 applications °ach year. 
Several factors are carefully considered in the selection process, 
including past school records and reading and mathematics scores. 
In order to ensure diversity among entering students, consideration 
is also given to etnnic background and residential location. 
Approximately 700-750 students ar* admitted to the school each 
year. 

Murry Bergtraum High School has a diverse student population. 
There are students at the school from each borough in the city: 
38 percent of the students are from Brooklyn, 31 percent from 
Manhattan, 19 percent f rc-r, the Bronx, 10 percent from Queens, and 

2 percent from Ricb^nd. Eighty-five percent of the students 
attended public eKnentary *nd intermediate schools. The others 
have come from private and parochial schools located throughout the 
five boroughs. 

Students are required to complete a program of coT'zge preparatory 
studies with a strong emphasis in business education and skills 
t> ining. Yheir preparation includes 4 yea's of study in English, 

3 years of social studies, 2 years in labcvatory science; at least 
2 years in mathematics; a year or more of computer languages or 
foreign languages; typewriting, and aesthetic arts; and 3 or more 
years of study in 1 of the followirg "concentrated" areas of inter- 
est: 

• Accounting • Secretarial Sciences 

• Computer Science • Securities and Finance 

• Marketing 

Field-Work Internship experience is required of all graduates. 
C;°lege acceptance rates arc among the l.ighest in the city; Zj 
percent of its graduates choose to further their education im- 
mediately; many others work for a year or 2, then return to higher 
education. 

School Enrollment: 2,943/885 boys; 2,058 girls 

80 percent black and Hispanic; 12 percent 
Oriental; 8 percent other 

The observations at Murry Bergtraum occurred in two classes of English, 
economics, computer s ri ence, and secretarial studies. Our project team of 
three members began observation on Wednesday, Octcjer 16th, and observed 4 
consecutive school days concluding on Monday, October 21st. The team then 
interviewed students (five from each class), each of the teachers observed, the 
principal, and the schooTs advisory board chairman. All interviews were 
completed October 21st through October 25th. 
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Data Cooing and Storage 

A considerable quantity of data was collected thrcugl „ the course of the 
project. These data were of a qualitative nature being comprised of observer 
reports of classroom events and comments made by students, teachers, or admin- 
ist Hors during the course of (unstruci Jred) interviews. The data were 
entered onto an IBM PC and formated for analysis by a text retrieval progrvvn 
entitled SUPERFILE. 

All 01 the data were coded by the field staff using the codes listed in 
table A-3, For example, portions of interview or classroom events that 
were relevant to the subject—collaboration among students on homework -were 
coded 14-00. Many of the data entries had multiple cedes assigned .0 them. 
The purpose of the codes were to serve as keywords for SUPERFILE, Additional 
keywords included class subject, interviewer name, school, intervivi* typj 
(student, teacher, and so forth), and class level (see figure A-l). To get a 
listing of all comments by Seniors at Cincinnati's School for Creative and 
Performing Arts, we would request SUPERFILE to retrieve data entries with the 
ke> ords 43-00 (higher education plans), SR (for seniors), and SCPA (for the 
high school name). 



TABLE A- 3 



TOPIC AREAS AND 
DATA COLLECTION CATEGORIES 
FOR INTERVIEWS AND OBSERVATIONS 



!• Relating to Others 



1- 00 

2- 00 



3- 00 

4- 00 

5- OC 



6-00 



Qualities or achievement* essential 
Qualities or achievements essential 



to be in leading crowd 
for: 



2-21 - peer respect 

2-22 - school administrator respect 

2-23 - teacher respect 

2-24 - guidance counselor respect 

2-25 - parents respect 

2-26 - respect of leading crowd 

2-27 - importance placed on each of the above groups 
2-28 - getting a good job 
2-29 - promotion on the job 
Friends in school 
Friends career plans 
Subgroups in the school 
*5~21 - share of friends In various groups 
5-22 - vocational students 

5-23 - socializing and conferring in other subgroups 
Parental career expectations 



II* Homework 



7- 00 

8- 00 

9- 00 

10- 00 

11- 00 



12- 00 

13- 00 

14- 00 

III. Grading 

15- 00 

16- 00 

17- 00 

18- 00 

19- 00 



20- 00 

21- 00 

22- 00 



Extent of assignments 

Nature of assignments 

Student Interest level 

9-01 - Student choices on assignment 

Extent homework is completed 

Extent of help students get on homework 

11-21 - parents 

11-22 - students 

11-23 - teachers 

11-24 - others 

11-25 - rules and expectations 
11-26 - actual practice 
Procedure for grading homework 
Reward 8/ penal ties for doing homework 
Collaboration among students 



Factors influencing grades 

Relative importance of factors influencing grades 

Extent to which a curve determines grades 

Extent to which grades are public knowledge 

Value placed on grades by students 

19-01 - Feelings about tests 

19-02 - Feelings about grades 

Parental influence on grades 

p eer influence/resptxt concerning grades 

Grading system fairness 

22-01 - Cheating 
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IV. 



Group Work 



23- 00 Structure of group work 

24- 00 Typical composition of group member* 

25- 00 Duration of group 

26- 00 Strategics used to deal with group dropouts 

27- 00 Strategies used to deal with group standouts 
2fc-00 Visibility of individual student contribution 

29- 00 Individual sharing of group responsibilities 

30- 00 Motivation derived frcm fellow students 

31- 00 Rewards for group effort 

32- 00 Distribution of rewards 

33- 00 Development of group norms 

V. Classroom Recognition 

34- 00 Chances for recognition 

35- 00 Kinds of recognition/rewards 

36- 00 Basis for giving recognition/reward 

37- 00 Level of competition among students 

38- 00 Peer reaction to recognition 

39- 00 Qualities of a student that teachers feel are important 

39-21 - tries hard 
39-22 - Improvement during year 
39-23 - knowledge of material 
39-24 - follows directions 
39-25 - helps others in class 
39-26 - very smart 

39-27 - interaction and participation 

39- 28 - self-discipline 
'0-00 Effects of putting additional effort into studies 

40- 21 - ways teachers learn about it 

40- 22 - consequences 

VI. After Completing School 

41- 00 Determinants of success 

41- ° 1 - types of jobs 

42- 00 Determinants ot success 

42- 21 - types of Jcbs 

43- 00 Higher education plans 

v<r I. Locus of Control 

44- 00 Impressions about ability to control what happens to them in the 

future 

45- 00 Ability to control what happened to them while in school 

46- 00 Control of ability to get a good job 



in jobs after completing school 

in jobs recently and/or currently held 



ERJC 



7 8 86 



Classroom Environment 



47- 00 Rules and expectations 

47-21 - Appearance of classroom 
47-22 - Appearance of teacher 
47-23 - Classroom decisionmaking 
47-24 - Appearance of student* ' 

48- 00 At tent ivenes 

49- 00 Discipline 

50- 00 Stiient participation 

51- 00 Attendance tardiness 

51-01 - Maintenance 

School/Curriculum Environment 

52- 00 School-vide recognition, e.g., honor society, PT0 recognition 

53- 00 Individuals' choice of classes 

53-21 - when choices vere made 
53-22 - why choices vere made 
53-2? - vho influences choices 

53- 24 - advanced classes 

Other 

54- 00 Teacher actions /unique instructional strategies 

54- 01 - Time on task 

54-02 - Interacting vith observer 

54-03 - Teacher conferring vith other teachers 

54-04 - Time off task (other subjects) 

54-05 - Time off task 

54-06 - Teacher planning 

54-07 - Teacher conferring vith students 

61- 00 Student presentations 

62- 00 Quality of vork/discussion 

63- 00 School cohesiveness 

63-01 - Importance in classroom 
63-02 - Importance to students 
63-03 - Teacher mentions unity 

63- 04 - Teacher informal interaction vith students 

64- 00 Racial issues 

64- 01 - Feelings about prejudice 

64- 02 - Teacher mention of prejudice 

65- 00 Mention of college 

65- 01 - Teacher relating class events to college 
65-02 - Student* relating his/her effort to college 

66- 00 Personal qualities of student 

67- 00 Motivation 

67-01 - Students feelings about it 

67-02 - Reasons given 

67-03 - Effects of motivation 
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68- 00 T-S interaction 

68-21 - Teacher encouragement 

68-22 - Teacher initiating student leadership 

68-23 - Teacher shoving trust 

68-24 - Teacher respect 

68-25 - Teacher becoming involved in personal problems 
68-26 - Teacher initiates humor 

69- 00 Risk taking 

70- 00 Self-esteem 

70-21 - Teacher praises student achievement 
70*22 - Teacher praise (general) 
70*23 - Praise from peers 

70-24 - Student acknowledgement of teacher prais" 
70*25 - Negative comments from teachet 

70-26 - Negative comments from one student to other students 



ERIC 



80 



88 



Observation Note Format for Keywords : 

Code 1/Code 21. . -/Observer/School /0/C1ass Name 

Interview Comment Format for Keywords : 

Code 1/Code 2/. . •/Interviewer/School/I/Rebpondent Type 

wherp Code i = codes from table A-3; variable number of codes per entry 

Observer (Interviewer) = NCRVE Staff Member initials 

School = UN - Westerville North 
WS - Westerville South 

SCPA - Cincinnati School for Creative and Performing 
Arts 

MB - Murry Bergtraum (NYC) 
MHEB - Mt. Hebron High School 

Class Name = CHEM - Chemistry (Westerville) 

CS - Computer Science (Murry Bergtraum 

DRAMA * Drama Lab (SCPA) 

ENG - English (Westerville Bergtraum) 

GERM - German (He* Hebron) 

GOVT - Ame 'can Government (Westerville) 

HIS - Hiscory 

MATH - Mathematics (Mt. Hebron) 

PSYCH - Psychology (SCPA) 

PHYS - Physics (SCPA) 

SECR - Secretarial Science (Bergtraum) 

Respondent Type * SOPH - Sophomore 

JR - Junior 

SR - Senior 

UNK - Grade level unknown 

TEACH - Teacher 

PRIN - Administrator 

GC - Guidance Counselor 



Figure A-l. SUPERFILE keywords other than topic 
areas and data collection categories 
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APPENDIX B 

FAIR DISTRIBUTION RULF IN A COOPERATIVE ENTERPRISE 

Suk Kang 

This appendix considers a fair distribution rule in a cooperative enter- 
prise such as a team of students participating in a cooperative learning 
situation in which rewards go to the group rather than the individual. It is 
assumed that (1) N agents are participating, (2) the total output of the 
enterprise is a function of the sum of nonnegative efforts contributed by the N 
agents, (3) the distribution of the output is determined by each agent's 
relative effort level, and (4) the effort level of each agent is observed 
completely by the other agents. 

It is postulated that the fair distribution rule needs to satisfy the 
three conditions. Denoting the effort level by the ith agent by Ei and the 
share of the output by oi, the conditions are written as 

(1) Completeness 

N 

I S, « 1 for all E • , i=l,.,N 

i = 1 J 1 
o * 

(2) Symmetry 

S, = f{E. t E 1 ) where E 1 = r.E- 

j*i J 

(3) Homogeneity 

f(aEi, aEi) ■ f (Ei , ei) for a > 0, 
where a is a positive constant. 

Equation (1) expresses the requirement that c-Jtput is completely distri- 
buted among all the agents; (2) expresses the condition that every agent is 
rewarded according to the same rule, that is, the share of the ith agent is a 
function of the effort level nf the ith agent and the sum of the efforts by all 
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the other agents; and (3) expresses the condition that a proportionate increase 
in efforts does not change shares. 

One rule that satisfies conditions (1), (2), and (3) is the one considered 
by Sen (1966): 

(S) S. = c(E./E) ♦ (l-c)(l/N) 

N 

where E is the total effort, E = I E-, and c is a constant. 

j=l J 

The rule (S) covers the complete merit rule, = E^/E; the complete 
egalitarian rule, S i * 1/N; and the intermediate cases between the two. 1 

It shall be shown that the rule (S) completely characterizes the fair 
distribution rule when the number of agents is more than three and the function 
f is differentiable. 

Theorem 

When f is differentiable and N > 3, the distribution rule (S) is the only 
rule that satisfies the conditions (1), (2), and (3). 

Proof 

Conditions (2) and (3) imply that the share S i can be written as » = 
gft^, where ^ is the ith agent's share of effort, t i * E^/E. Condition i 
then implies that S i = 1/N when t i * t /N, that is, when all the agents expend 
equal effort, the output is evenly distributed. 

The share of the ith agent is written 

(1) S. = g^) » 1/N ♦ h^). 

Condition (1) requires 
N 

(2) l h(t.) * 0 for all t = (t y ...t N ), t^O, Et. - 1 



Uising the distribution rule (S), Sen (1966) derived the optimum value of the 
weight c in the symmetric Nash-type equilibrium. Under the assumption that 
total output is a function of total effort, the optimal weight is given by the 
elasticity of output with respect to effort level. 
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and 

N 

(3) I zhit^/U. = 0 for i = 1....N. 
j=l J 1 

Equation (3) can b2 rewritten as 

N 

(4) I h'(t.)t. = h'(t ) i = 1....N, 
j=l 1 

where h' is the derivative of h. Equation (4) implies 

(5) h'(t.> = h'(tj) for all t. and ty i,j = 1.....N, it, = 1. 

It is straightforward to verify that when N > 3, the first derivative of 
h is constant, and so h is linear in its argument. One can. write u[tA as 

(6) h(t.) = dt. ♦ e, 

where d and e are unknown constants, arc! solve for d and e. 

Substitution of (6) into (2) yields 
N 

(7) d z t, ♦ Ne = d - Hs * 0, 

j=l J 

Then, 

g^) = l/N ♦ dt^ - d (l/N) * (l-d) 1/N ♦ dt r 

Q.E.D. 

Remark 1 

When N = ?, equation (4) is reduced to h'f^) = -h'(l-t^); therefore, any 
'unction that has the property h[t^) * h(l-t i ) = 0 satisfies the fairness 
conditions. This is because when N = 2, there is only one degree of freedom in 
the shares of effort. 

Remark 2 

The theorem does not necessarily imply c > 0. That is, more effort by the 
ith agent, when the effort by others is constant, mav result in less share by 
agent i. For an economically meaningful and fair rule, c should be restricted 
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tj be nonnegative. Furthermore, it may be reasonable to impose the condition 
that when effort 1 evel of the ith agent is nonnegative the ith share is also 
nonnegative. To satisfy this requirement c needs to be less than or equal to 
one. Thus, the requirements of fairness considerably restrict the form of 
distribution rules. 

References 

Sen, Amartya K. "Labour Allocation in a Cooperative Enterprise. " Review of 
Economic Studies 33 (1966): 361-71. 
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APPENDIX C 

INCENTIVE SCHEME AND DETERMINATION OF EFFORT 
IN COOPERATIVE ORGANIZATIONS 
Suk Kang 

This appendix considers the relationship between incentive scheme and 
effort level by the agents in a cooperative enterprise. The term Enterprise 
may represent a cooperative learning team, co-op, labor-managed firm, or any 
other organization in which total output of the production unit is determined 
by the effort level of agents, where the marginal productivity of the agent 
depends on other agents 1 effort level, and where output is completely distri- 
buted among the agents. The problem addressed in this appendix is that of 
determining the distribution rule that induces agents to choose the optimal 
level of effort. 

It is posited that (1) the welfare of the organization is measured by the 
unit of effort, (2) each agent voluntarily chooses his or her effort level, and 
(3) the individual agents 1 effort levels are visible to all the agents. Fur- 
thermore, it is posturted that there is agreement among the participating 
agents that the distribution of the output should be determined based on the 
individual agent's effort level and that the distribution rule must be fair. 
(See appendix B). 

In the neoclassical theory of distribution, workers get paid their margin- 
al contribution to output, and the pro/it (surplus) is taken by the owner of 
the capital. However, in the cooperative production model, the total product 
net o to be fu? ly distributed among the agents so that wher. the production 
function is not homogeneous of degree one in effort, the neoclassical distri- 
bution rule will not be a solution to our problem. 

In deriving the optimal distribution rule, I will consider a special case 
in which total output is determined by the sum of effort by the agents: The N 



agents are identical, and the production function is in Cobb-Douglas form. 
Denoting the effort level of the ith agent by E^, total effort by E, and 

total output by Y, the total output is written as 

(1) Y » E*, where IE, » 1 and 0 < B 

i*l 1 

The share (ratio) of the ith agent to total output is determined by the ith 
agent's effort level and the sum of the all other agents 1 effort. Limiting the 
distribution rule to the one that satisfies general requirements of fairness 
(see appendix B), the share of the ith agent is written as 

(2) S. » (1 - a ) 1/N ♦ a(E i /E). 0 < a. 

Equation (2) may be interpreted in the following manner. The distribution 
rule is a wefltjhted average of the egalitarian rule, « 1/N, and the complete 
merit rule, * E^E. It is assumed that the marginal cost of effort per- 
ceived by the agent is constant and r<y~ttr. across the agents. Given tech- 
nology (1) and distribution rule (2), the objective function of the ith agent 
is written as 

(3) S,Y - cE it 

where c is the marginal cost of effort, c > 0. Differentiating (3) with 
repsect to E^, the first-order condition for the maximum is given by 

(4) aE B " 1 ♦« (B-lJEjE 8 " 2 ♦ (l-a)/N x BE*"* - c. 

Since, by assumption, the agents are identical, the equilibrium level of effort 
must be symmetric, that is, the optimal levels of effort are equal across all 
the agents. 

Denoting the equilibrium value of effort by E*, substitution of E* into 
(4) yields 

(5) E* * [cN/WN-lJ+B)] 17 ^" 1 ^ 

Hence, the equilibrium value of effort is (1) decreasing in the marginal cost 
of effort (c), (2) increasing in a (see [2]), and (3) decreasing in the number 
of agents (N) . 



We consider the optimal distribution rule given the technology and the 
number of agents. Equation (5) shows that by raising " " effort level, the 
total output will increase. However, more output does not necessarily imply 
higher level of social welfare. This is because, at the higher level of 
effort, the marginal return from extra effort, which is expressed as the 
increment of output, may be less than the marginal cost of effort. 

In order to derive the optimal level of effort, the social welfare of the 
organization at a symmetric equilibrium is written as 

(6) V = E* - cE * (NE*)* - cNE* 

where E* is the value of effort corresponds to a symmetric equilibrium. The 
optimal value of E* is then given by the following equation: 

(7) E* = (c/B) 1/(B " 1} /N. 

Equating (7) and (5), we can see that the optimal distribution rule must 
satisfy 

(8) a = b 

Note that 0 is the elasticity of output with respect to effort and the 
distribution rule can be written as 

(9) S i = [(Y/E - 3Y/3E) E/N ♦ (aY/aEJE^/Y 

or 

(10) S.Y = (Y/E - (3Y/aE))E/N ♦ (aY/aEJE-. 

Equation (10) implies that ith agent's share (in the unit of output) is the sum 
of the two components. The first is the marginal product multiplied by the 
effort level (see the second term of [9]), and the second is the profit per 
worker, with profit being the output that not distributed according to the 
individual agents' effort level. Although the optimal distribution rule in 
(10) is derived for the Cobb-Douglas case, it can be shown that the rule (10) 
applies to the general functional forms. (Miyazaki 1979). 

Condition (8) implies that if the distribution rule is such that a < B, 
agents underinvest their effort, and 1f a > &, overinvestment of effort 
. occurs. 
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